I have three or four pipes that I bought unfinished, all of them in various states of finishing themselves. The older ones are thirty plus years old. I never gave handling any thought. Obviously don't handle them with anything on your hands that would stain them. Otherwise, handle them all you like. Actually, handling them all around will probably spread the oil from your hands evenly and make them finish evenly, but I'm not sure I'd bother thinking about it. Watching unfinished pipes develop is one of the joys of pipe smoking and I highly recommend it to anyone. Mine have tended toward a beautiful deep walnut glow, which is completed after about ten years, but handsome long before that.