The way brace height affects the life of a bow is commonly misunderstood, but it is fairly simple.
Consider a bow that is unstrung. The limbs have zero potential energy in this state.
Now consider a bow that is braced. You had to put work into it to string it, right? Now there is potential energy stored in the limbs. The amplitude of that potential energy can be felt in the tension of the string. If you brace it higher the potential energy goes up, and bracing lower makes it go down. The initial string tension scales with the potential energy stored in the limbs at a given brace height. The thing to understand here is that potential energy is ONLY zero when unstrung.
Now consider a bow released from full draw. As the string resets to the brace height the limbs move forward toward their original unstrung position but are stopped short by the string. Energy equal to the potential energy stored at brace is exerted into the string, some of that energy is used up when the string stretches (here the string is acting like a shock absorber) but the rest of it travels as a wave from the string grooves and the string loops in toward the center of the bow and string respectively. String vibration is a result of wave interference when those waves collide in the string. Hand shock is partially a result of the same but in the riser (in addition to some other mechanical causes of hand shock if the point at which these waves meet is in the grip, and/or their magnitude is high enough, it will cause hand shock).
The waves traveling down the limbs, and the subsequent vibration stress the material bonds in the bow (think modulus of elasticity). Past a certain extent it starts breaking them, once too many of those bonds are broken the bow breaks.
Conclusion? The higher your brace height, past a certain extent, the more damage you do to your bow because the magnitude of the wave goes up.
So no, lowering your brace height won't hurt it.
Another way to think of what you are doing by lowering your brace height is that the limbs are actually not being pulled as far back when drawn because the string is longer,