Sorry for the hiatus and necro-post, but life's been busy.
Normally I'd let sleeping threads lie, BUT I figured i should post my solution in case someone else is wondering "how can I bow hunt AND still be able to draw my side arm if I need it?"
The solution I found for me was to use an AEE Cavelier tab, with the spacer on, and not use the finge loop. If I need to, I can drop it, but I have noticed zero loss of performance so far, and I can wear light weight gloves, with it, which will help with the whole hyperthermia prevention thing.