I'll second what Bjorn and slivrslingr said. I've hunted for many years without camo, mainly in wool plaid.
There are probably more hunters that have hunted without camo than with the stuff, considering that camo was not used much prior to 1957. The first pictures, I remember of Fred Bear with camo where at the time of "Little Delta Hunt". Howard Hill, Art Young, Dr Saxton Pope, Will Compton and many others did not use what we refer to as camo. Ishi hunted his whole life without camo, and mainly without much clothing.
Animals are very good at picking up movement, camo or no camo. I have hunted animals that saw me, and because I avoided looking in their direction, or making eye contact, they did not spook and would continued to feed, or do what they were doing.
Sometimes, I wonder how the industry go so big. It sure made a hit with varmit hunters, turkey hunters and bowhunters. Obviously, it has its advantages when hunting turkey, varmits, or when calling elk.