Some pipe makers (Nachwalter, Marks, et al.) actually finish pipes with olive oil on occasion. I've never had problems with it going rancid. However, it will darken a pipe with a light or "natural" stain, so beware of that. On darker finishes, it won't matter much.
I've used it for stems as well, but Obsidian Oil is MUCH better. Brebbia stem polish is good, too (just takes a little elbow grease).