Here's a few things about photography that I've learned from my mistakes over the years.
Fit and Comfort - First of all, find one that fits your hands and that has controls that you are comfortable with. If you have big hands that could be a problem. A slightly bigger camera makes it a lot easier. Go with a good brand because if you ever want to move up to a nice SLR, brand compatibility means control compatibility. You'll already know how to basically use it. I bought my kids the same brand in point and shoots as my high end equipment. Guess what? If I hand my son or daughter a big SLR, they can use it with little or no explanation necessary.
Budget - You could spend $100.00 or $1000.00 on a small camera depending on what you want in features. Usually, you have to spend MORE to get better manual features. From what you said above, I'd bet you just want good and simple. You can get a great camera for under $250.00 these days and some $150.00 models are almost as good.
Strength - A strong housing is a must if you take it out into the field. Worst case senario is that you drop it from a tree stand. That will pretty much trash any of them. Bouncing around in a fanny pack can be rough too. Especially if you have a fall. If it feels cheap and flimsy it probably won't hold up in the field. Neoprene cases are small, inexpensive AND they float too! Its good insurance.
Waterproof - That's nice but a Ziploc freezer bag (heavier ones) works fantastic AND helps to keep it clean too. Pros use them all the time over lenses. It also helps to protect it in that fanny pack also.
Lenses - Glass - Period. Plastic, although better in today's models is still plastic. Ask the salesperson about it and if they don't know, look in the owner's manual as for the specs on the lens. The downside will be higher cost.
Viewfinders - Some have them as a backup to the big screen on the back. I like them and they work in all types of light. Imagine your on a hunt of a lifetime, screen is broken or damaged and you want some pictures of that 180 class buck you just recovered. Even worse, its COLD! Most LCDs don't work in the cold! No viewfinder equals guesswork on framing the picture.
Megapixels - Go for more than less. Normally you just need 6 MP for blowing up to an 8x10 for that trophy room wall. Hold on, unfortunately, you didn't frame the shot very well (usually the person you handed the camera to didn't) or there's some object that you don't want in the picture. Now you have to crop the picture on your computer or at the photo shop. Now, without higher megapixels, the picture gets grain (or noise as its now called) and you loose definition of the picture. Plan for the worst and things will be fine.
Flash - First, make sure it has one. That can definitely cut your photo session short when you recover that buck after the sun goes down. Second, opt for a model with the better flash if possible.
Camera Shake - Your pictures will be blurred unless you hold the camera still. Impossible with a small camera. So you need to set the ASA or speed to 400 or faster. Again, higher megapixels helps here.
Tripods or Monopods - Not needed so don't let a salesperson talk you into one. However, there are two gadgets that are very nice to have with you especially if you're hunting alone and want a picture of you and that buck. Both are very small and take up very little room in a fanny pack. One is a snake type twist around thing that you can wrap around a limb, screw your camera onto (there should be a tripod hole on the bottom), set your camera, frame your shot, set your timer and get in the picture. The second is a small rod with a wood screw on one end that goes into a tree, limb, fencepost or whatever and the camera attaches to it the same way as above. Both are inexpensive and can put you in the picture instead of behind the camera.
I didn't mean for this to become a long photo lesson, so just follow some of these guidelines and you shouldn't have any problems in selection or use. If you have any questions, feel free to PM me anytime.