I think the general rule is to let it rest a day at least. The longer the better, I suppose. I have seen people say that giving a pipe an hour or two to cool down and dry out a wee bit is enough, but I'm not sure if that's enough.
Remember, way back when everybody smoked pipes, people would only own one pipe, or maybe two - they'd smoke the thing all day, every day, with rests overnight.