Ditto all the above; and I'd throw in another possible cause: We all know that good grouping is a result of "consistent" form.
Part of consistent form is gripping the string with the same finger pressure for each shot. While the pressure by each finger forming your 'hook' might be different, as a unit, that pressure has to be the same from shot to shot.
If, as you've said, you occasionally lift the arrow off the shelf, this indicates different finger pressure by one or more of your fingers from the times that you don't lift the arrow.
I have purposely applied different pressure with my index finger and have been able to get different vertical impact points, to prove this 'point' to myself!
Sometimes, too much thought process interferes with what would be our normal, consistent, form that can take place when we are totally focused on the impact point.IMHO