It doesn't seem to affect a recurve much because they are cut closer to center, but extensive testing that Dan Quillen did years ago proved to me that a longbow is made to "move out of the way" at the shot. A quiver slows this movement enough that it will affect the shot.
That has always stuck with me so now while I use a bowquiver, I always take it off once I get in the tree. I think it's mostly physcological but heck, most of what we do is.
Like a few guys have already said, if you're gonna shoot with a quiver on, practice with one on.