If you have to cut off some material, your taper tool is cutting the tapers too long and the blade needs to be adjusted.
Also, need to be careful, slow and easy, in cutting the taper. Pretty easy to get a less than perfect taper with a hand held tool, particularly on hardwoods and softwoods like Doug fir that have a big difference between winter and summer grain hardness. That's why I use a sanding disk to grind my tapers.
Regardless, can still get them on straight. Use a hot melt glue and spin the arrow and adjust the point if necessary before the glue hardens. Or, reheat the point a bit and adjust until you have it on straight.