Deer get used to human scent in small areas that are surrounded by people. You need to hunt the wind as well as possible, but the swirls are going to get you busted a certain portion of the time. That's just a fact of close in hunting. My small place has occupied lots on all sides, so human smell is a constant. However, they seem to know very well which smell is distant and which is close enough to be a threat, just as they do with noise.
The best oak trees on our place are in my son's back yard, and he regularly has multiple deer around the house, and they are not overly frightened. However, further into the woods, they get real skittish real quick; it is as if they realize that the odor is in a location that is out of the ordinary.
I have noticed that, as a general rule, prevailing wind direction in the morning favors certain stands, while evening favors others. Also, set numerous stands hunting a given area, thus having a little help dealing with the wind.
In short, small populated areas can be tough to hunt. In my experience, cover scents don't seem to make much difference, but I still use them fairly often.