I'm pretty sure shanez is correct on this one. I've never had allergic reactions in all my life -to anything- Not even hay fever or anything seasonal... until I was stung by bees back to back in one week a few years ago. The second time, my hands swelled up like literal balloons.
Since then, I developed "oral allergy syndrome" to plums and cherries. It is basically just an itchy mouth. On rare occasions (because I still eat them), I have broken out in hives. Goes away after an hour or so, but it seems to be related to whether they are fresh off the tree or not. I can eat them dried, in jam, cooked, or older (most of the time) without issue.
A doctor prescribed me an epi pen, in case I one day kick over into a crazy reaction, which he said is possible at any time. Just be ready for the chance it can happen, was his advice.
So, unless I am unusual, I am a living example of how allergies can shift and change by degree over time. But, there is also medical literature to back this up. I got interested when it impacted me. Strange stuff, allergies.
Take away point: If you have an unusual reaction to something, I would be extra cautious the next go round. Way it was explained to me is that you don't necessarily have an anaphylactic shock the first time. It may just be a little unusual, with extra swelling or itchiness. It can build up until one day, it's just one toke over the line and someone is taking a pocket knife to your windpipe to try and save your sorry ass.
Not sure if red virginias can do that, but I believe those who say they are impacted. Makes perfect sense to me, even if it is from the same plant as others they can tolerate. My experience is similar with plums and cherries.