You are right in that middle ground where sometimes you do everything right and hit where your looking, and sometimes you don't but can't figure out why. Am I correct? I've been there.
It's a frustrating place to be, but the only way to get past it is first make sure your arrows are spined right for the bow your practicing with.
Second, pick an anchor that's comfortable and stick with it! I can't stress that enough. Don't play around with different anchors because it only confuses your eyes and your brain when it's trying to "Remember" where to focus. Pick an anchor like the corner of your mouth that feels natural and keep doing it repeatedly over and over and soon it will become second nature.
Repetition is the secret to getting good groups. Your eyes will learn to adapt to where you need to point your bow hand over time of repeated shooting.....Lots of shooting!
My last piece of advice is until you get your instinctive shooting skills honed, stick with the same bow. Don't switch back and forth with a different bow, that just makes things worse.
You will see what I mean when it all finally comes into focus because it's like sitting in a dark room by yourself and all the sudden somebody flips the light on. Boom......it all comes together.
I was once where you are my friend, and these things are what helped me and I still can't shoot worth a crap!
You'll get there, just keep flinging those arras.