Dean Torges wrote an article on this called "seeing spots" which explains your problem well. But, it seems his new site doesn't have this article?
It has something to do with how your brain sees an object. So that, when you draw and shoot your instinct will allways put that arrow somewhere near the outline of the object. i.e. the edges of the deer, the edges of a block target, the edge of the ball, the edge of ping pong ball, the edge of the gnat. When It comes down to it, aim small miss small.