I did try sheet rubber cut to the right size using biopsy punches, and it worked as well as the leather washers so go ahead and try.
I have smoked several pipes with that system (A fair number of prewar Royal Comoys had them) and truly think they smoke much better with the apparatus removed entirely.
I did have a box of #4 washers that I sold last year, and they occasionally pop up on e-bay.
Doc