Ok, nerd alert! The question isn't really about speed vs. energy because an arrow's energy is a function of it's speed. Actually it's not even a question of bow weight (bow weight does affect speed and energy but some bows of the same poundage can shoot harder/faster than others due to design...so it's an inconsistent variable) or momentum (momentum relates mostly to a fixed arrow velocity and it's mass...hard to quantify on an object that slows down quickly as it moves to it's end). It makes more sense to use kinetic energy (a measure of the energy of an object in motion) as a yardstick, even though that isn't perfect either. Still, a bow/arrow combo that delivers at least 30 ft. lbs. of K.E. should work pretty well for deer-sized critters, a broad guesstimate based on what seems to work in the real world for most bowhunters. The problem comes when folks get into the mindset that very light arrows shot from very light bows are great for target shooting so they must be ok for big game because the arrows seem to fly pretty fast. That MAY work...but only as long as the K.E. for an on-the-edge bow/arrow combo is enough to overcome the penetration roadblocks posed by less-than-perfect hits (bones and scapulas are alot harder to penetrate than soft lungs...doh).
So, for a bit of insurance in the event, however remote it might be, that we all don't make perfect hits all the time, it's best to stay away from 'ultralight' bowhunting tackle for bigger critters. It might work fine for perfect lung hits on stationary animals but for the average shooter under average hunting conditions that isn't always what happens. So, why risk it?
I suppose a better way to say this is, why not just let common sense have it's day.