At some point in your pipe journey, you start getting better at noticing what was last smoked in a pipe. When it's objectionable, people call it "ghosting", but we don't have a great term for when it's beneficial and reinforces the flavors and aromas we enjoy in particular blends... "seasoning", perhaps?
I'm not too worried about ghosting in the pipes I use for aromatic blends. I assume that whatever new I put in will clobber the previous residues in time.