I smoke them both with and without filters, and I also use both types (paper and balsa). I have found the filters are awesome when used with the appropriate tobaccos.
I am highly vulnerable to tongue bite, and filtering helps.
I use the paper filters for aros. These tend to mute the flavor a bit, absorb moisture, and in general make the smoke smoother. I have had otherwise unsmokeable aros rendered downright pleasant through the use of a cob with a paper filter.
The balsa filters are good for any blend which you don't want turned down a notch. Contrary to what some say, I don't believe the balsa filters change the flavor at all and can't figure out how they could. The Smoke never passes through them at all but around them. They are good for sopping up excess moisture and thereby reducing the temperature of the smoke, which can reduce bite. You don't lose any flavor or complexity that I can detect, so these are good for Virginias and Latakia mixtures.
Just experiment with different pipe/tobacco/,filter combinations until you know what you like.