I usually take it to the point you have there. After my cleanup sanding to remove any tool marks, if there are cambium pockets like that left I’ll leave them as it looks neat. If not I won’t fuss over it. If who I’m building for wants a clean back, I’ll sand it clean.
Kyle