On anything that shoots projectiles---bows, firearms, etc.---any change to the apparatus itself (or the projectiles) will result in a difference. Everything has to be consistantly the same for consistant results. That's not saying you can't change anything. It's just that once you make a change, you need to make adjustments to bring the point of impact in line with your point of aim.
If I had to guess, I'd say it's your nocking point. Break out the stethoscope (your bow square) and check it out.