I have had trouble with this myself over the years, and have read many different perspectives on the causes and cures. There is a current series of posts a few posts down, "Target panic reality check," by Joel Turner that has helped a lot of people. The book, "Instinctive Archery Insights," by Jay Kidwell has also helped a lot of people. I think there can be more than one cause, so you just have to try different things until something works. Clearly, if you're not overbowed, and you're physically capable of holding at full draw without undue stress, then it must be a mental issue if you can't do the same thing when you want to shoot an arrow.
I had a bad case of target panic for a while, and then seem to have licked it, but find myself backsliding from time to time. These are the things that I do to keep myself on the straight and narrow:
1. Every day, when I'm by myself, I draw the bow and hold for 5 seconds before I shoot, timing myself with a metronome. I repeat this five times, both left and right handed. This helps because it makes a normal 2 second hold seem short by comparison.
2. If I find myself rushing the shot, on the next shot I will draw and hold for a count of three and then let down. Then I will draw and shoot my normal shot. This normally gets me back in the right rhythm, but if it doesn't, I will draw, hold, and let down again, and keep repeating with alternate shots until my natural rhythm is reestablished. I'll do this in the middle of a tournament if I need to.
3. During every shot, in addition to focusing my visual attention on the target, I open my awareness to what I'm feeling in my body as I draw the bow. This focuses my attention in the here and now while I'm making the shot, rather than worrying about something in the future, like whether it's going to be a good or bad shot. It also shuts up the little voice in my head trying to give me useless instructions on how to shoot the bow. The more I can feel all the little movements my body is making during the shot, the better the shot turns out. The more I think about what I should be doing, or try to do any particular thing during the shot, the worse the shot turns out. It's hard to just "not think," but if I have something to substitute for thinking, like feeling, then that can occupy my mind with something useful.
These things have kept me from slipping back into target panic, but if I had a full blown case, I might need stronger medicine.