It usually doesn't matter how dry the tobacco is, as water vapour is a natural by-product of burning tobacco. It is interesting that you say this happens when you are not smoking the pipe. The same thing happens with me. My theory is that water vapour is condensing in the stem and draft hole. The upward pressure of drawing on a pipe keeps it there. But when you stop drawing, or if the moisture becomes too heavy, then it falls back in to the burning tobacco and the sizzle is that moisture turning to steam.
One way to minimise this, and it takes practise, is after every few draws of the pipe, is to give a very gentle blow back down the stem. This helps move moisture down into the burning tobacco to vaporize silently, before it builds up to the extent where it causes sizzle. This also helps keep the smoke drier, and helps smoking to the bottom of the bowl. Don't blow back too hard, do it gently, or you will blow ash out like Krakatoa!