Here's what I did, and it worked. But I didn't use it much, so I have no hunting success with it to speak of...
I bought a thin, inexpensive fleece-type jacket, then cut a lot of it away. The right arm, gone at the shoulder. Chunks cut out of the chest area. But I left a little area over my left chest/shoulder to hold the ghillie material out of the path of the bowstring. And I cut the back side of the left sleeve off, all the way to the cuff. I left a strip above the elbow, and a strip below, to hold it together. So basically, I had a long left sleeve with three giant rectangular chunks cut out of the back side of it. Then I pulled that over my Rancho Safari long coat. I pulled the ghillie material through the holes I'd cut in the fleece thing, so it would still accomplish its purpose for concealment. When I wore the thing, I put my short leather armguard on in its normal position, over the modified sleeve of my cut-up fleece jacket, and over the arm of the long coat. I put it on in such a way that when I stretched my bow arm out to draw my bow, the modified fleece jacket sleeve pulled tight against my arm (from the shoulder) and pulled all the ghillie stuff out of the way. YMMV.