I'm not defending, I'm simply pointing out, different people have different moral values. Some of us, I well understand, feel morally superior to others. Hefner made a lot of money challenging the moral ethos of the day. People resent having their moral values challenged. Moral judging is one of the tools we all use in determining who we will or will not socialize with. I much prefer judging through the prism of legal? Not legal? The vocal/public moralizing. judging of people and their sexual preferences? I'll leave that to others. Of course I have my personal thoughts but, they, in interest of peace on the board, are best left unwritten.
I admire Hefner simply because he lived his life as he saw fit. I tend to admire that in a person. His moral values really do not enter into that judgement.
An aside: If one accepts an invitation to an orgy, isn't participation expected (I have no personal knowledge of the proprieties of an orgy). I've not seen any assertions stating that people were forced to attend parties at "the mansion." Sports fans are often escorted out of a venue for improper behavior. People are often escorted out of bars because they were not drinking/s[ending money. Before theaters became social venues unruly customers were often asked to leave.
A person just hanging around, watching the sexual escapades would be called a... voyeur. A lot of people would consider a voyeur to be a deviant. Of course some of the participants might be exhibitionists and welcome a voyeur. Damn! It'd be rough work being security at the Playboy Mansion. Who to toss? Who to let stay? Who's offended? Who's gratified? Damn!