I am also VERY colorblind and it is a drawback, no question about it. Having said that, I've wounded and also made several killing shots on deer that I wasn't able to recover, hunting with a bow and with a gun. When that has happened I have, on a couple of occasions, retired my tag for the season, so to speak, but most of the time I don't.
Nothing goes to waste in nature and my inability to recover a particular animal, though unfortunate for me, isn't for the varmints that can use the protein for the winter.