Just to get it straight. I don't recommend shooting a bow that isn't comfortable for you, at all. If your not accurate with a bow, consistently, then its not a hunting bow.
But that doesn't mean that it cant be one, or that its not at least a workout bow. If anything, I go down in weight to hunt, shooting in the 70's to hunt squirrels, rabbit, elk, deer, carp, whatever. Bows that are too heavy for me to be consistently accurate, as accurate as most guys if not more, don't get shot any further then 7 yards or so. With LOTS of time spent point blank just drawing to full draw, releasing smooth. This process will help you get comfortable, as comfortable as you are at 46, with much heavier bows. IF you want to do that.
In the end, its only going to make you MORE accurate at any given weight to be stronger or more capable at heavier poundages.