Young Marine it looks like the Little favorite gains 3.5 pounds per inch going from 31 to 32 inch draw.
That's very good for a bow that's only 54 inches in length.You would probably notice it.But it may not bother you at all.Especially if you are not overbowed.You might want to order one a few pounds lighter than you typically shoot.With a 32 inch draw you are gonna have a ton of power even in a lower poundage that you typically shoot.
I have had a few bows that stacked a bit but shot fine for me when I was drawing 30".3.5 pounds an inch at 32 inch draw aint bad at all IMO.Five or 6 pounds an inch would be very noticeable and you would not feel as smooth,easy or as comfortable reaching anchor that last inch.
I have hunted a lot with longer bows and they can be more of a hassle in some hunting situations as you have expressed.I have tried a lot of short bows and have found some that are very agreeable in terms of smoothness,stablity and shootability.
Hope that helped?