I have a small study at ground level in our home, and while I usually smoke a pipe outside, it is nice to be able to sit at the desk at night with a pipe sometimes. I go the window fan route, a single fan leaned against the screen to blow out. I usually smoke Latakia blends and only one time have I had the family comment on smell. I figured it was coming in either through the ventilation duct or under the door. So now, when I want a pipe at the desk, I push a rolled towel against the door bottom and close the HVAC vent, in addition to putting my fan up. No more complaints, and even though I take the fan down and close the window when I'm done, I've never smelled a whiff of smoke in there the next day. Not bad considering it's usually a Latakia blend.
As far as noise, this fan is VERY quiet. I can still easily hear the rain, and the creek running by outside, through the window. The fan isn't disruptive at all.
Point is, I don't think it takes much oomph at all, just the smallest amount of air movement to keep the smoke from building up and sticking to the inside of the room. I think if you allow that to happen while smoking, it'll be a lot harder to get rid of the smell afterwards.