The folks who intermittently hunt and fish from year to year have many reasons. They will say no time, lost hunting land, lost friend, doing other things, rained opening day, etc. Of course these days lost job is going to be a big one, and rightfully so because it a rare person who needs money only for a hunting license when they decide to hunt.
States are trying to get better at marketing hunting instead of just taking for granted that hunters and fishermen will keep coming. In 2009, hunting licenses in 34 states were lower -- some by 20-30% than they were in 1999. To inspire us to hunt every year they try to remind us how fun, rewarding, and necessary hunting is. Again, it is difficult for most on this site to imagine anyone inspiring us to hunt because we are hard-wired for it.
In 2004, the Outdoor Writers of America reported that only 25% of the children of hunters and fishermen were taking up these pursuits.
It would be really interesting to see states narrow the gap between resident and NR license prices. Even if they were equal the NR would still spend much more on travel expenses than residents. However, you can bet to narrow the gap the resident fee would have to come up a lot as the NR license went down. Residents would really squawk. In fact, much of the pressure for higher nonresident fees comes from the resident who doesn't want the NR competing for available hunting space.
Others have written above. The people of the state owns the wildlife, not the nonresident and not the landowner. The wildlife agency is required (in most cases) to provide for the perpetuation of all wildlife species in the state. Things that cause the NR license to be more expensive include: residents want this to be the case, the agency figures the NR can afford or is willing to pay more, and that's been the standard procedure.
I understand and support this system and these strategies. I've been part of these kinds of decisions. Frankly, I'd love more of a chance to draw rare tags such as Shiras Moose, Bighorn Sheep, and Goats but they aren't available in my state -- heck, according to current odds it will take my 51 more years to draw one of the new KY Elk tags! But like someone wrote above, if I want to improve my chances a little bit I could move to another state. In fact, when my son-in-law gets reassigned (Air Force) to the U.S. in 1.5 more years he intends to head to the Pacific NW. They are holding one of my grandchildren "hostage" so I might have to follow them.