Not much. I've used both, and truthfully, I can't tell any performance difference. Theoretically, a 4 fletch should be a bit slower due to drag and a bit quicker to stabilize due to drag. Perhaps if one had the right equipment, one could actually measure these differences.
In practical terms, the only difference I have noticed is I use one more feather with a 4 fletch than I do with a 3 fletch. Since I'm cheap, I have opted to stay with 3 and save one feather per arrow.
You will get many opinions on this topic, but it will kind of boil down to what you like best, what looks cool to you, and ultimately what gives you the most confidence. If you think you shoot one better than the other, go with that one.