Go up and read Terry Greens post a couple of times...JC's too.
They are two of the best shooters I've ever seen personally shoot. Terry has amazing form and a flow to his shot that is an envy to everyone who sees it. JC looks like he snap shoots because he is so quick but he just flat hits everything he shoots at. He's into the consistant "minute of squirrel head" accuracy that I dream of someday achieving. Ironically, both of them spend about zero time at anchor (as pecieved by a bystander) so that may be some kind of answer to your question in itself.
Personally, I had to get away from shooting quickly because it caused me to start snap shooting and short drawing and that is a habit that is hard to break. It's also closely tied to the dread "IT" otherwise known as target panic. I now shoot with a very deliberate anchor and hold for two to four seconds. The length of time seems to depend on how well I'm able to hold my bow arm solid. If it wavers off target, I need to hold a little longer to get back on. I believe the guys who shoot quickly often have a good solid bow arm, therefore they know they are on target when they hit anchor and so can release quickly and still hit their mark. After all, why waste time looking at the spot if you know you are on it when you hit anchor? I, with my somewhat under control target panic, have difficulty shooting quickly because I'm afraid of going back to my old habbits.
As a mediocre shooter who has spent a lot of time trying to correct a "short draw-can't hit anchor" problem, I have to recommend continuing on as you are with the solid anchor and concentration on your form. Just practice good form and alignment until it is ingrained then start simplifying your mental side of the shot and work on getting to where you can just concentate on the spot. Speed will come with time if your brain works that way. If it likes to shoot slower and more methodically, you'll still be able to speed things up a little but maybe not like some guys.
Shoot the way that works, don't be afraid to try something new and be honest with yourself about the results.
p.s. One of the best pieces of advice I ever got on shooting was from a mulitiple time winner of the Texas State Longbow Shoot Kieth Bain who said simply, "point the bow at the target and then DON'T MOVE IT until after the shot."