Here in Long Beach, Ca. there are several coffee houses with outdoor seating where I smoke a pipe. Since I've gone from latakia blends to burleys, which actually smell pleasant to most non-smokers, I haven't had a problem. I was at a coffee house last week and a couple walked by, obviously not noticing the pipe in my hand, and the woman said "That's nice. I wonder what kind of incense that is?" The funny part (to me anyhow) is that here boyfriend / husband said "I don't really know!"
Seriously, stick to real coffee houses (not Starbucks which has banned smoking outdoors), drop the latakia and if that doesn't work just come on down to Long Beach.