As Blacktail noted, your broadheads might already vary by more than 5 grains. Depending on how you sharpen them, there may be as much as 10-15 grains difference in weight between your heads, and they're likely less than 155 grains. Too, unless you're weighing your arrows to within a few grains, they can vary by a lot more than the 10 grains plus or minus that the vendors say they group them in. In short, your arrows might have a physical weight range approaching 50 grains, and only a little bit of that is likely due to variation in broadhead weight.
Unless you have a good grain scale and weigh the arrows, or better yet, the individual components, you won't really know what may be causing the less than perfect flight with your broadheads.
One thing I do to try to arrive at equal physical weight arrows is weigh each individual point and shaft and put the heavier heads on the lighter shafts and vice versa. That can reduce a physical weight range of 20 or more grains to almost no difference.
Regardless, if your broadheads are only 5 grains lighter than your field points, and if your arrows are closely matched in physical weight and spine, you simply will not notice the 5 grains difference.
Be sure your broadheads are mounted perfectly straight. Broadheads that aren't mounted perfectly straight contribute to a lot more erratic flight than a minuscule difference in weight.