Shoot and shoot and shoot your equipment. You will gain confidence as you gain experience. Overbuilt design red oak selfbows are not going to be as fast as modern fiberglass recurves. You have to determine if it is fast enough to kill effectively.
A well designed and built selfbow of 50# should do the trick. A poorly designed and built bow may not.
It is likely that the bow is more capable than you are
I believe this was your first bow, right? My first 3 bows were not hunters. My fourth one was. I still shoot it occasionally but never killed a deer with it. My 5th bow was a hunter also and I killed the deer in my avatar with it. It was a sinew backed static osage pulling under 50#. It was nowhere near as fast as my glass recurve of similar weight. If I had to guess, I'd say it was shooting in the low 140 FPS but it did the job.
Deer are relatively small in broadside. It doesn't take much to shoot a sharp broadhead propelled by a heavy arrow through them. A turkey could be killed with a kid's bow IMO
Above all, have fun and keep shooting. Come August or so, be honest with yourself and decide.
I'm going to be a one selfbow man this year in preparation for a September elk hunt. I still need to make my bow though.