I had a Hoyt Gamemaster for a while with a sight, and my plan was to do pretty much what you are planning. Since you hold the bow in a vertical position when shooting with sights, and canted when shooting trad, the sight is more or less out of the way when you don't want to use it. I've also tried shooting gap for longer shots in competitions, and trad for shorter shots, which some well known archers have done successfully, according to what I've read.
I don't do that anymore. I decided that in the limited amount of time I have to practice, I should just focus on what I like the best, which is trad. I'm certainly not as good as I would like to be in Trad, and as long as I feel that way I'll probably focus on that.
But what you're proposing should work; no reason not to do it if you want to.