You won't notice much at 15 yard, but at 20 you will start to notice a difference in trajectory. Beyond that, there will be a significant difference.
As Jason said, nothing is free. If you change weight, you will change spine, so you will have to either use a longer arrow or one with a weaker spine. Depending on your bow, you may also notice more noise with the lighter arrow.
On the positive side, I dropped about the same amount of arrow weight you are contemplating, and I shoot more consistent groups out to 25 yards (that's about as far as I can get in my driveway). You won't have any trouble killing deer with either setup.