There are more plastics used for stems than some may realize. Acrylic is but one, but the most common. Chris Askwith is known to use a polyester stem material which is more comfortable than acrylic. Juma is a favorite of mine which seems to have the best qualities of Vulcanite and acrylic. Bakelite is another very hard plastic, but one which oxidizes, albeit slowly.
In the end, I care most that the stem is comfortable and has a smooth and appropriately-sized draft. In general, I found that more often in Vulcanite stems than those made from any other material. Many acrylic stems are preformed, chunky and unrefined, like those found on nearly every new meerschaum pipe. But you'll find many better Italian pipes made with very well-made acrylic stems.
I am most likely to buy a pipe with an acrylic stem when I don't need to clench very hard, which for me is usually a jaw-hanger. The other situation where a hard acrylic is fine is at the other end of the spectrum -- a large or extremely long pipe that is too uncomfortable to clench, regardless of the stem material, so I end up just holding the pipe anyway and might as well enjoy the no-fuss care of acrylic.
One great thing about plastic-stemmed pipes is that they're pretty much the only ones that I leave out on display. Nearly all of my rubber-stemmed pipes are tucked away in dark drawers, shielded from oxidizing light rays.
I also have become a fan of horn stems. They are actually very comfortable and in the hands of a good carver, like Peter Heeschen or Chris Askwith, they can be shaped beautifully as well. They can be prone to toothmarks, but thankfully, it's not much of a problem for me.