Its not an allergy (unusual reaction to substance) many woods are just plain toxic! In increasing toxicity; Walnut, osage, yew. Many other "exotic" woods are also poisonous. These toxins can also be cumulative - your reaction will increase after continued exposure. Yew is so toxic that the bark was the original source for a cancer chemotherapy drug Taxol (yew is Taxus Latin name). As Micky the Ferret used to say, "Osage dust makes me feel like I have a cold, yew dust makes me feel like I'm going to die!"
The fine dust from sanding is particularly dangerous. Wear dust mask or respirator and avoid exposure to dust as much as possible. I try to do most of my cutting and shaping with sharp edge tools and limit sanding and sawing.