You do what works best for you. Myself, I enjoy ramrodding my musket as I contemplate the cosmos in my pipe.
I’ve learned to tweak the packing of different blends in my pipes. If I’m tamping even 1/4 as much as I’m pipping, I’m annoyed beyond enjoying it. Constant fiddling isn’t for me.