Here are a few ideas
1. You could be getting some planing. Check them when the wind is dead calm. If no planing then and you are using very big feathers try smaller feathers. If wind isn't an issue Check their alignment and make sure they are consistent arrow to arrow. I line up blades with the feathers (assuming you use 3) so they all fly consistent.
2. Also the broadhead is much longer than the similar field point which can cause some weakening of spine. Try a lighter snuffer and if it flys correctly use thenm or You could try shortening the arrows taking 1/4" off at a time and see if they fly true with the heavier BHs.
3. If you are using small feathers, switching to a larger 3 feather config or using 4 feather config might stiffen then up enough if it is a spine issue.
4. If you use the point of the arrow TIP for aiming and are RH the longer arrow will cause right and low shooting. Use the end of the shaft as your reference point not the tip of the point so length is consistent.
Everything I can think of.