I think you're dead on. I currently have 4 bows; 3 recurve a and 1 R/D longbow in various draw lengths, draw weights, and bow lengths. All mine are one piece bows, but one is a Super Kodiak, which has a similar feel to a three piece, and I have shot several three piece bows that belonged to others. I've also shot different weight arrows from most of them trying to see how it affected speed, noise, etc. I haven't fully settled yet on one setup that I think is perfect. I have a 43# recurve and a 47# longbow that both shoot great with the same arrow weighing between 550-600 gr. they aren't speed demons by any means with that arrow, but they're quiet and the arrows fly great. I have a 53# recurve shooting approximately 530 gr arrows. It's very noticeably faster, but it's also noisier. I've also shot a 500-525 gr arrow from the longbow. With that arrow, it nearly matched the speed of the 53# recurve. It's a little quieter than the recurve, but noisier than it is with the heavier arrows. There is a very noticeable difference in the feel of each bow. I can shoot any one of them very well, but if I shoot with just one for a long period of time then switch, the new one feels weird to me. But it only takes a few days of shooting it for it to start feeling right. If I then shoot it for a while and switch back, the one that was so comfortable originally will feel weird. There are also other factors to consider. Different lengths fit better to different hunting styles. The weight can also be important for hunts involving lots of walking. Then there's personal preference, which may not have any rhyme or reason behind it. And of course, aesthetics. Doesn't affect performance, but it's still important to some people. I think the key is to get a setup you like and stick with it.
Matt