I've had the same experience as you, started out with aluminum, and still have a very healthy respect for it, but...
I just love the look and the feel of wood. I don't think it's nearly as durable, consistent, and it's kind of a pain in the butt...but I just love the feel of it leaving the string.
I don't make my own arrows, but I do enjoy wieghing, spine testing, tapering, straightening, finishing, and fletching my own shafts.
It's definitely a labor of love. I can think of very little to justify the wood shafts besides the fact that I darn well like them, but that's good enough.