If you don't have a clean release, i.e., a tendency to pluck a little, it will send the arrow to the right, for a left hand shooter. A too stiff arrow bouncing off the side plate will also go to the right. Of course, for a lefty, the arrow impacting to the right indicates a too stiff arrow. But you already know that. It is where the arrow impacts though, not its orientation in flight, that is the primary indicator of its stiffness, so don't put too much stock in a tailing in-flight shaft appearance. Fletching does have a stiffening effect, if for no other reason than it adds weight to the rear of the arrow.
Given you want to use full length arrows, if your 400 arrows are currently impacting right, you should be able to move them back to center by moving the adjustable arrow rest out. I think that should work, but if doesn't, you may need to go to 340s, but definitely don't need to go any stiffer than that.