If you only get two of you to start with, pick anywhere and set up a chess board between you... even if you don't know how to play. Then, if someone comes up to complain, you'd be two guys smoking pipes and playing chess. They'd be more likely to ask you physics problems than ask you to leave. :
:
Seriously, I hope you get up a group. Pipe clubs are a lot of fun, and you get really diverse folks to hang out with, which always makes for great conversations, especially where tobacco and pipes are concerned.