Yes, they "learn" to avoid danger but theres a new crop of birds pretty regularly. That same tom may not come in again, but hormones often get in the way of common sense (not just with turkeys).
If I were targeting him, I would go without decoys or add a jake and relocate a bit. If you have a good population to choose from, the next tom might charge right in.
I've had days where birds got to the set-up, didn't like it, and left the scene in a hurry. Then a couple hours later there are toms knocking over the decoys.
I hunted a completely virgin property that was loaded with birds. As expected, the birds had no fear of the blinds or decoys. A couple years later it was leased to an outfitter and hunted hard. After a couple years of that they were pretty wary of the blinds. Of course there are always jakes to fall back on!