I have been building bows all month.. same problem. Once I finish sand with 320, I just use the clear coat and it all goes away. If you really want to take care of before the clear, keep moving up the sanding chart.. 400grit, 1000grit.. etc. Like you are polishing.
But if you get your finger wet and wipe it on the bad spot and the bad goes away, the poly will do the same thing to it. That is how I test to see if I sanded enough!
Good luck