I don't think there is single answer for all the areas experiencing a decline in deer. So all the ideas expressed are probably factors.
The area of Indiana I hunt has experienced a significant decline in the past couple of years. I talked with some of our DNR folks and here are the key points they have made.
Generally speaking the weather across the state has really benefited farmers this year. Also a good mast crop, so the deer numbers for the whole state are likely up.
Ups and downs of the deer population are very spotty with some counties way up and others down.
There have been outbreaks of EHD. Years of extended dry spells seem to increase the population of the midge flies that transmit the disease and/or concentrate the deer around scarce water sources making it easier for the midge to get to larger numbers. Most deaths happen in late August and early September before a frost kills the midge. EHD outbreaks can be very localized. With very high mortality rates, a local group of deer can simply disappear. Hunters in such an area can see a dramatic change in sightings in just 3-4 weeks.
A lot of people around me are still out of work, or working many fewer hours than before. Lot of folks hunting for food that may not have other wise hunted.
Poaching is up likely for the same reason.
Despite large numbers of anterless permits, the buck to doe ratio is still very low in many areas as a lot of hunters still fixate on a trophy. I've suggested, and no one in DNR listens, they should consider saying for the low ratio counties, one has to tag a doe before they can get a buck. Low rations make it harder for the herd to recover in down areas, and hunting is not as much fun. For example, rut sign is minimal in low ratio areas.
The population seems to recover, but it can take a long time. I used to do a lot of hunting in SE Ohio. Lots of deer, good ratios, plenty of sign in the 80's and 90's. The area I hunted was written up in a couple of national magazines and the herd went to pot in the early 2000's. I think to over hunting. Just now seeing it recover.