The picture shows two fletched arrows almost touching and two bare shafts right on top of each other, that's very good accuracy. If you had the right combination of spine, length, and point weight both the fletched shafts and bare shafts would all be together. Like you said your arrows are showing weak. I like to have my bare shafts and fletched impacting together, others like a slightly weak reaction...but I think that's probably a little weak for anybody, especially at only 20 yards. With your shooting skills I would do your bare shaft tuning at 25-30 yards if possible. Get bare and fletched grouping together at 30 yards and you know you are well tuned.
Rather than cut your shafts back I would try lighter points just to see how weak your shafts are. Even if you don't want to use the lighter points it's cheaper than cutting arrows and would give you a lot of information. I would throw the lightest point I had on there and see what happens. In other words, try to make them too stiff, if you can you know somewhere in the middle is optimal. Sometimes it saves a lot of time to make a big change rather than sneak up on it. Changing point weight is easy and reversible if it wasn't the right move. If you find you only need to go down 25-50 grains, cutting might work or you could play with side plate thickness or brace height. If you need to go down to very light points, down 100 or more grains, it might be time to think of stiffer shafts.
What are the details of your arrow now, spine and length? I have several recurves in that draw weight range (50#-59# at my 29" draw length) and I can shoot the same 31" .340 carbon out of all of them just by tailoring the point weight to each bow (150-250grs.).