a photo would allow for better advice...
that being said, various pipes I've restored such as some of my Disney World Pipes and my Birth year 1975 Dunhill had a slight gap in one side as if the stem was crooked. The stem was not loose and slid in straight as it should but when it met with the shank there was about a hair gap on one side.
I took fine sandpaper 1000grit and a flat piece of wood to create a firm and flat sanding surface, then sanded around the touching edge and slightly up the sides so as to turn the angle just slightly; almost imperceptibly. really just watching the surface to indicate progress of sanding. Finally finished with a flat sand across the whole surface to ensure a flat plane. My pipes are now flush 360 degrees around the shank.
Of course it all depends on the pipe and the nature of the misalignment.
"The true soldier fights not because he hates what is in front of him, but because he loves what is behind him." - G. K. Chesterton