If you are hunting, speed is not the holy grail; weight is much more important. Like Moe said trad bows generally have a pretty small variation in speed. Speed doesn't increase much with decreasing arrow mass, but inertia does decrease significantly with decreasing mass. Mathematically the equation is:
inertia = mass x velocity
If you decrease mass but velocity doesn't change much, inertia will actually go down. Conversely, if you increase mass and velocity doesn't change much inertia will actually go up.
IMO switching to carbon to increase speed isn't worth it. Having said that, switching to carbon to get more durable arrows is certainly worth considering.