What are you wearing now??? (specificly)
I wear wool. For whatever reason, my legs rarely if ever get cold. I work outside all winter long with a pair of bluejeans on my butt & nothing else. I wear upper body layers, as the weather dictates. For hunting & work, I have over a dozen 100% wool sweaters I have picked up at the GoodWill store over the past years. I'm probly fortunate as wearing wool against my skin,does not bother me at all!!! For a "base layer" as some would call it, I wear a thin Merino Wool sweater, or a wool "yuppie sweater" sleeveless sweater vest, instead of a T-shirt. Then, 1 or 2 sweaters over that, then my favorite ODgreen&black plaid Woolrich hunting shirt. Or, if it's REALLY cold!!! A caped wool Mackinaw. I'm usualy comfortable down into the single digits.
Remember, You are wearing LAYERS! So, they should be sized accordingly... I'm a BIG guy,(ok,short & fat,whatever) An XL sweater is snug on me, so I have a few XLs for under,and some XXL & XXXL for over. Most folks buy their hunting clothes to "fit" instead of buying them oversized, to allow for underlayers.
Another thing... "Cotton KILLS"!!! We have all heard this before... Blah,Blah,Blah.....
Cotton as a base layer,is bad because it holds the dampness against your skin & saps your body heat. Cotton OVER your wool clothing is one of the best and CHEAPEST windblockers you will find!!!! I have worn 2 heavy wool sweaters,(against my skin) with a XXXL cotton camo shirt,(unlined) and been warm & comfortable in windy/rainy,windy/snowy, weather with temps down into the teens. The cotton keeps the rain/snow and dampness away from your skin AND keeps the warmth of your body heat(trapped in the wool)against your skin!!! (Think "waterboarding"... Air will not travel through a wet towel) My appoligies, if that offended anyone! But,it's the best example I can think of now...
Pick up a couple of 100% wool sweaters from the thrift store and give it a try!!!
Good Luck!!!!!