The reason that arrows seem to tune stiffer when having a bow mounted quiver is do to the extra weight of the bow.
As you release, there is a tendency for the bow arm to move due to the release of pressure. The heavier the bow, the more effort it takes to get the bow started and less likely to affect the arrow before it is clear of the bow.
The heavier the bow, the less the quiver will affect the above.
I've gotten so used to shooting with a quiver on that I wouldn't consider shooting the other way. I think that for most that want to shoot with a bow quiver, all they need to do is keep shooting and not go back and forth.
It only takes 1 arrow for something new to become "IT" good or bad. It's hard to say which arrow will be that 1. Could be a day for some, a week for others… or much longer... but it will happen if you want it to. Stay the course or change your mind and go without it.
BigJim