If effect, your neighbor is upset that you're in the way of him shooting deer coming off of your land.
Do not compromise with someone this wrong.
If he wants deer to bed on his land, he needs to do some habitat work to improve his own land.
Instead, he wants to use your land as his private bedding area and he doesn't want you hunting there, on your own land.
If anything, he should move his stands if he's relying on hunting the bedding area's on your land, that you are already hunting.
This would be like if you had land on a steelhead river and the neighbor upstream told you not to fish because you had first crack at the fish moving upstream and he didn't want to getting in his way.