As you can tell by the replies, it depends...
Sometimes tamped ash will choke the ember and the pipe won't burn properly, that's generally when I get rid of it. I stir just the top, ashy part, and invert the bowl. Most of the ash falls out, then most times I'll blow across the top of the bowl...