Last year in preparation for elk hunting, I asked Allen Boice of Liberty Bow what his thoughts were on this topic.
He thought I would be fine with my 50# bow. But he could tell I wasn't convinced.
He didn't want me to buy a new bow for potentially one hunt so he sent me his personal 55# bow, same length, same model.
In shooting both bows (with appropriate arrow for each) I found the 55# bow penetrated my target only 1/4" deeper than the 50# bow. But, all bows are different.
At some point, reducing weight will get too light and I don't know at what weight that is.
Personally, I don't want to be marginal so the two bows I bought this year are both 55# with my Stickflinger having a set of 49# back up limbs for the future. And a Liberty longbow, of course.
At 65 years old, it does take effort on my part to shoot 55# bows, but I think it is worth the it.