It's the sulfur in the vulcanite that oxidizes. Most of the time the sulfur is only partially oxidized, that's why it can be a pain to get off. A method like oxyclean finishes oxidizing the rest of the surface sulfur making it "soft" and allowing it the be removed much easier. Once all the surface sulfur is oxidized it can be removed pretty easily with a mild abrasive like a magic eraser sponge or barkeeper's friend.
The problem with using a heavy grit paper is that you basically sand away the top layer of the vulcanite exposing more sulfur to light, heat, and oxygen which starts the oxidation cycle all over again. It's better to "pre-oxidize" all the surface sulfur using the oxyclean method (or some other similar method), rubbing away all the oxidized sulfur using a mild abrasive, and then only using very fine grit papers to finish the stem. This will smooth the surface that has had the sulfur removed instead of exposing more sulfur. This involves a LOT more work but it's a much better method than just going after it with a heavy grit sandpaper or (just as bad) brown tripoli on a buffing wheel. Just my two cents on the subject.