Bryan-
Now... this is on private land mind you, but the scenario is fairly the same. I have set up a ground blind and hung a trail camera on the front in the late morning. When I have checked the camera I have had photos of deer that afternoon.
Here is my opinion. If a deer shied away from every suspect lump in the woods, they would NEVER get anywhere. If you set up next to a well traveled route that has steady traffic I don't think you would halt that activity. I wouldn't plop down right in the middle of the trail and expect no reaction but otherwise I think you would be OK. Also, I would make good use of the see through mesh if your blind has it. My experience is that deer are more afraid of the giant black hole an open and uncovered window poses than they would a large camo lump that suddenly appeared. You will definitely get some head bobs and hoof stamps but with no movement and little scent I think they will quickly go back to business as usual.
I sit some hunts with a ghillie suit and stool as is suggested and it works, but like you said you are out in the wind and the cold. A blind is significantly more protection. With a ghillie and stool you also have to play the wind a lot more. A blind lets you contain your scent more.
If you can incorporate your blind to blend in to the natural surroundings it helps. I wouldn't go to the trouble of brushing it in (might be against public land rules to cut trees anyway) but setting up next to a blow down or in a gap in a thicket makes it look more natural.
Below is a photo of a blind set up of mine. Everything is still free standing. I didn't cut any of the brush to place around the blind I just snuggled it into what was there.
Good luck with the hunt!
OkKeith