up here it's gone below zero and it's weird how used to cold you can get. Last week 20 something was horrible now it's fine.I would think that it would depend on how cold. Being here in Alabama, 40F is what my body has adjusted to as extreem cold, but then this last week it has been in the teens, which lets my body know that there are colder temperatures than I have never experienced.
Maybe if the pipe was way below freezing when you light it up, there could be a problem. But, I would think that there would have to be several situations set up just right to make this cause a problem for the pipe.