Both my parents smoked cigarettes when I was growing up - indoors, outdoors, in the car, everywhere. They stayed with my wife and I in our first apartment for some months in the '70s. When they left, we had all our curtains and soft furnishings professionally cleaned because of the strong, lingering, stale cigarette smell.
I packed my pipes away, sadly, when we were expecting our first. I got back into smoking cigars when both boys finished their university courses.
I smoke outdoors only, and not around children. And most definitely never around my 3 year old grand-daughter.
Wearing a smoking jacket and a smoking fez/cap may be one way of minimising tobacco particles and smoke clinging to your clothes. At least your wife will be impressed you are going that extra mile.
And look debonair at the same time.