I have had no issues with or without cake. However depending on the blend and how dry it is could be a factor. Dry tobacco is imo not only better for flavor and smokeability, but makes a nicer harder cake as well. Moist aromatics tobaccos, (Carter hall is really my only example and only blend close to this category) have a tendency to make a very weak cake, that could flake off easily. Could cause burnouts if smoked without caution. I used to take whole 1/8 in chunks out of my cake, leaving massive crators, and causing me to ream back to wood.
Now days I only worry about any of the small leaf pieces barely hanging on to the chamber.