I'm always considerate of others, but I don't remember feeling self-conscious about smoking outside in public (except for the fact that I'm still fairly young). I've never heard a negative comment about the pipe, at least where I could hear it. I even smoke around the kids, babies included. My kids, I mean.
I think it may depend a good deal on where you live and what kind of situations you choose to smoke in. For example, a flea market would involve a lot more close contact with others than, say, out walking the dog or whatever.
Still, there are many people, I'm sure, who want and expect smokers to feel ashamed and uncomfortable.