ethan....
You kind of answered your own question.
If you are not using the netting...then absolutely...you must wear dark clothing...cover your hands...face....etc.
However, the netting covers a multitude of sins. You are correct...turkeys don't mind the holes. (Movement in the holes bothers them.) But...I still use the net. It gives me a great deal of confidence when I begin my draw. I still wear a face mask and gloves...because the encounters are usually pretty close range....and I don't want the slightest bit of movement rousing a turkey's attention.