Realistically, you might shoot over the deer on the first shot, and it might hang around or return in a few minutes for a second shot, so you don't want to bother looking for the arrow right then. You might hit the deer on the second shot, but find that it needs a third shot to finish it off. If you only bring 4 arrows, that leaves you with only one extra arrow in case you need it. Plus whatever blunts you want to shoot to pass the time.
I assume that if you were that confident that you would finish the deer off with one shot, you probably wouldn't be asking the question.