[I tried a clay tavern pipe and for me it left a bad taste in my mouth and the coolness of the smoke actually made my teeth hurt. Maybe this was just a breaking in period. The clay itself actually tasted bad. Probably need to put a bit on it.]
A real good clay should have no taste by itself and does not impart any taste to the tobacco. As a matter of fact it is my understanding blenders use primarily clay pipes in their testing of the blends for this exact reason. Probably your tavern clay was made with a poor material. Try another one before giving up, I would recommend Lepeltier also because they are double walled and so not so hot to touch as the tavern and also because they have have a vulcanite stem so more civilized in holding.