I like the guideline that's out there: 10 grains per pound you shoot. I think my current set up is in the 12-13 per pound.
Orion is right: you could stack deer with your current set up. Beyond the penetration advantages of heavy arrows, other advantages of heavier arrows come with more quiet shots, and I'd have to imagine that the heavier arrows will translate to longevity of one's bow. Some arrows I've seen being shot out of trad bows (350 grain out of a #55 bow, for example--the bow sounds like a .22 going off) is too close to a dry fire for my taste.
You are on the right track. I'm sure that the penetration advantages of heavy arrows can get to a point of diminishing returns, but you aren't anywhere close to that point. Keep it up.