I have made them for bamboo arrows. We have a dowel maker. If you run some osage thru your saw and get it down so it fits in a drill press you can sand them down that way, I have done that also.
BUT, stange as it sounds, I have had trouble with osage nocks breaking. I even soaked them in supper glue to try and strengthen them and they have broken. I have had good luck making them using cherry dowels and oak dowels and they look good. I still coat the wooden nocks with super glue before putting a finish coat on them. If you want some pieces of cherry dowel to turn down, (using a drill press) PM me with your address and I will give you enough to do 1/2 dozen of arrows if you want.