I've had great luck bowhunting during the general gun season, even in areas with lots of hunters. If you'd like to see my last rifle season archery buck, click on my handle at left. I shot him about 2 weeks from 15 yards with a Pronghorn longbow.
Wearing orange is not a problem. After years of hunting in orange, I am convinced deer simply do not pick it up, and it is about the right shade to blend into the environment I hunt. Lots of folks dress too dark.
Safety concerns are demonstrably not justified. Just check the hunting accident stats for your stat and see how many shooting accidents are not self-inflicted or members of the same party.
It took me awhile to get used to having other hunters around. I've had folks set up right next to me, or sit a short distance away and have lunch, talking all the while. I've gotten used to it. They come and go, and the deer come by in between.