What's happening when you over-tighten is that the threads are "jumping", that's why it goes loose. Essentially the ring is stretching outward and the thread on the lid "jumps" the thread on the jar.
You mostly see thread jumping (generally speaking with anything, not specific to jars) with soft materials like plastics, or in cases where you're using mixed brands where the threads might not have exactly the same pitch or may not have exactly the same diameter, or where even though the thread pitch/TPI is the same, the threaded length is different for each.
In other cases with harder materials (metal into metal, like a metal screw into a piece of metal, or metal and plastic), you tend to instead strip the threads rather than jump them.
If the lid rings are old, they may be more malleable, or the threads could be worn down, or the lids could be misshapen (out-of-round).
It's never a good idea to over-tighten anything, especially when there's gaskets involved. May seem counter-intuitive, because "tighter means tighter", but over-crushing a gasket can deform it and cause an open area that will leak if you exceed the gasket's squish tolerance.
