Jesse, are you shooting off the shelf? Are you using 3 fletch or 4?
Some will say it doesnt matter, but regardless, ive had clearance issues where a quill/fletching is bouncing off the shelf. If you are using a 3 fletch configuration, turn the nock so your fletching orientation is 12 o'clock (cock), 8 o'clock, and 4 o'clock.
Another possible answer to your problem, is to add a feather arrow rest. Many of them older bows used an add on rest, and the feather rest may just solve your headache very cheaply.