Dan, that COULD lead to imbalance(and shock) when shot by hand, that is, if you don't 'heal the grip' enough to mimic what the tree is doing.
Try taking the shim out, or out enough that the grip itself is perfectly level, then draw the bow on the tree and take note of any difference in how the hook tows the line, or not.