I have my concerns about it too. I haven't been able to find a whole lot of info on the possible downsides either. They're a problem here, but I don't know that poison is a good idea. Hunting them does very little to control the population though honestly. Our lease is around 2k acres, and they come and go depending on water. But I know we've trapped or shot over 100 in a deer season before and not even put a dent in them. And as the landowner's agent on this property I'm hesitant to let people I don't know hunt because you just can't trust folks unfortunately.
I know people that have let others come in and hog hunt for free on their place and they've torn up roads/pastures on 4 wheelers and in 4wd trucks, killed deer, left gates open and beer cans everywhere, let cows out or in pastures they don't belong in, and even stolen stands, feeders and cameras. There's just no easy solution.