I usually wear some lightweight liner gloves that I can shoot with (I use a tab).
I can dry these easily, and often have an extra pair in my pack.
Sometimes, I also wear waterproofed insulated leather mittens over the gloves, as well as over my tab folded back onto my palm. I take the mitten off of my right (drawing) hand to shoot, and let it hang from a homemade leash.
I made a 6 foot cord mitten leash that goes inside my jacket and comes out each sleeve. This length is my wingspan.I used fishing snaps on each end of the cord, and tied some small loops on the cuffs of the mittens to snap into.
This way I can slip off the mittens and let them dangle, and they don't get lost or fall in the snow/water/mud. They are right there to put back on, so it is easy to take them on and off. I do not find them to be in the way for shooting.
The leash system is handy for me as sometimes I need the mittens on and sometimes need them off, depending on what I am doing, how hard the wind blows, etc. This way I don't have to take off my pack to stow or get out the mittens, so I can switch back and forth as needed.
Once rigged up, I can leave the mittens attached to the jacket by the leash going through the sleeves, ready for use on the next outing, and not liable to be forgotten.