If I had access to a machine shop - I'd go about it differently... I'd use that idea, but with a longer belt so it would stay cooler and not gum up as fast.
Or, I'd just use a sanding drum. Actually, I woke up this morning thinking about this same thing! I have a 3"x3" drum that I was thinking I should make a pattern-bearing for.
I also have the Ridgid sander. So, I can rough-shape with that, then take it to the pattern-drum to finish it up. Hmmm..... Now I just need to figure out the bearing - shouldn't be too hard, as I have the stuff to do it. Yeah right! I know I can, it will be trial and error, though!
One other consideration - drill press bearings are not meant for lateral pressure, just vertical. If you push sideways too much, you can ruin the drill press pretty quick. Ideally, the center "bolt" would be anchored at the bottom, also. Does that make sense?