Joe, I don't know what your area is like, but if it is mature woods this is what I have observed : the years after logging, it became a magnet for deer in our area. The reason I think is the tremendous amount of new growth draws deer in, so I would not necessarily give up on it.
I lived the scenario of gaining permission to hunt some land over and over again. When my kids started hunting, I wanted a safe place to hunt without worrying about other hunters. I bought a 120 acres and paid for it over time. For about the price of a car, it hs been a great investment.
Perhaps you can not afford it now, but there are other ways. A couple of friends of mine got together with another guy and bought some hunting land together and then divided it up into smaller affordable parcels- 80 acre lots I think. They have enjoyed great deer, bear and grouse hunting for 15 years now.