Deer are conditioned to look for you in many places. While on the ground in my Ghillie, I was smelled by a doe leading a group. She immediately went to scanning up high in the trees and stomping around. She focused on one particular tree and it seemed she really convinced herself there was a hunter up there. She stayed just out of range, but I was laughing so much I probably couldn't make the shot anyway.
By the way, I've blamed UV before too. After wind, movement and having something behind you are number 1 and 2 to worry about though.