B-Dog,
I had the same problem. I shot all summer in a T-shirt, but when the cool weather clothes came on, my accuracy fell apart -- due to string slap on my shoulder/chest/arm. I wanted to use my ghillie-type camo and knew I in trouble.
So, I bought a camo YOUTH large fleece zip-up sweatshirt (I wear a men's large) on clearance for $8. Then I cut off the right arm neatly like a vest, then did the same with the left arm -- except I left a 4-inch wide strip of the arm that goes all the way from shoulder to the wristband, and left the wristband on. So, instead of a left sleeve, there's just a strip of fabric that covers the part of my arm where my clothing gets in the way, the full length of my left arm.
I use a short armguard, and when I put it on (pretty snug), I pull that sleeve-strip under it so that when I stretch my left arm out to shoot, it pulls that strip tight and even stretches a bit. This, combined with the tightness of the makeshift vest, pulls all of my clothes out of the way when I draw. It works perfect for me. I did it this way so that I could get as much benefit out of my ghillie suit as possible. (By the way, I had a buck at 30 yards last week [shot and missed, standing on the ground] and after I shot, he looked right at me and walked around, trying to make me out, but couldn't. I was standing in the open.)
I haven't tried it with heavy winter clothes yet, but if the vest is too small, I'll make a new one that is big enough to go over a larger jacket.