Feedback Loop Disintegration, Anyone?

[zanxion]

Imagine it like this:
1. You start to use AI
2. you find it good and you begin using it more and more
3. you get dependent on that AI for all your heavy thinking
4. some figure, political or not, manipulate that AI you trust to give you biased results
5. that figure eventually manipulates you
6. you, after unquestionably trusting that AI, never realize you are being used.

And there are other sidelines to it too. You replace much of your thinking to an AI, and perhaps after a generation or less it replaces the basic thinking. People no longer spend mental effort for the bare basics, and AI takes over that role. At the very beginning it was free, but now people must pay for using it, they must pay for something to do the basic thinking they no longer can do.

 

[Chris T]

I think you might be confusing training with indoctrination.

EDIT: I'm the spawn of a public school, so I don't really know the difference myself anyway. I did however have a Magic 8 Ball as a kid, and it cleared up a lot of confusion for me...it was proto-AI for millions of us.

True story:
Two kids in junior high were sharing a seat on a (public) school bus, looking out the window. One made a remarked about the moon, which was pretty high and still bright. The other guffawed, "You're so dumb! That's the sun! It's daytime, the moon's not out! Hey everybody, he just called the sun the moon!" The first said, "Then what's that?" and pointed to the sun through the window on the other side of the bus. The other boy got really quiet after looking back and forth out the windows several times.

I would say indoctrination is what the second boy exhibited when he said the moon's not out in the daytime. It's what he was taught. Critical thinking was what he was doing when he got quiet, trying to reconcile his prior accepted rules with his own experience. Training is what happened when the first boy said "what's that?" and pointed out the inconsistency.

Sadly, I have found that while some people don't have a need for training in critical thinking, a lot of us do. Without the training, some will just go along accepting the truth of whatever is told to them by those they see as authorities.

And - I can't resist throwing this in there- the unexamined life is not worth living.

 

[brian64]

Sadly, I have found that while some people don't have a need for training in critical thinking, a lot of us do. Without the training, some will just go along accepting the truth of whatever is told to them by those they see as authorities.

True...only problem is the training most people get is designed by those same "authorities"...and they're not interested in producing critical thinkers.

 

[Pypkė]

I’ll dissent here.
People don’t need to be “trained” to think critically. It’s the trained lapdogs that usually create the problems in the first place.

Critical thought can be transmitted to others. It's a matter of learning (i.e. training) like anything else that requires some discipline and study to pick up. Not talking about "trained lapdogs" not that you've made yourself clear about what you mean by that term.

 

[Pypkė]

So why are they still paying the corporate media talking heads millions to do the same thing?

Advertising has grown into quite the science.

 

[sardonicus87]

Thinking takes effort, humans are largely lazy. This is my biggest concern with AI.

 

[huntertrw]

The Internet and A.I., just two arms of what Mr. Orwell presciently termed "Newspeak."

 

[woodsroad]

I think you might be confusing critical thinking with indoctrination.

No, I am not.

 

[jguss]

People have picturesquely (and with great relish) celebrated the ephemeral nature of all things for centuries. AI is only the latest fashionable harbinger of doom.

No thing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away.

Or if you prefer the more obscure runner-up:

What powerful but unrecorded race
Once dwelt in that annihilated place.

 

[sablebrush52]

The topic of critical thinking is an interesting one. Last week I did a bit of research on what percentage of Americans practice critical thinking, and it is a practice. I thought that the percentage would be relatively low, maybe 20%. Boy, was I optimistic. The results of the various studies ranged from a low of 1% to a high of 4%. Explains a lot.

My father was a prominent aerospace engineer. How prominent you might ask? In the late 1970's AirResearch, the company at which he was the chief engineer, participated a trade show in Moscow. Engineering students were making the rounds of the various exhibits and when a group of them stopped at the AirResearch exhibit they suddenly got very excited. Soon there was a crowd in front of the exhibit. Turned out, Pop was mentioned in their textbooks for his invention of the foil air bearing, still very much in use in turbines. It was like he was a rock star to these students.

I spent a lot of time with Pop, listening to him as he went through his process for finding answers. He questioned and investigated assumptions, including his own. He had a remarkable clarity of mind. It probably didn't hurt that he was a genius.

What I learned from him was that process of investigating and questioning, and it's now automatic. That practice is what had enabled me to turn around a number of troubled productions in the course of my career, steering them away from the iceberg. That said, I'm not nearly as rigorous as he was, and I certainly don't have that massive number of cylinders under the hood that he had. But, the instinct is always active, whether I choose to pursue it or not.

 

[BingBong]

The topic of critical thinking is an interesting one. Last week I did a bit of research on what percentage of Americans practice critical thinking, and it is a practice. I thought that the percentage would be relatively low, maybe 20%. Boy, was I optimistic. The results of the various studies ranged from a low of 1% to a high of 4%. Explains a lot.

My father was a prominent aerospace engineer. How prominent you might ask? In the late 1970's AirResearch, the company at which he was the chief engineer, participated a trade show in Moscow. Engineering students were making the rounds of the various exhibits and when a group of them stopped at the AirResearch exhibit they suddenly got very excited. Soon there was a crowd in front of the exhibit. Turned out, Pop was mentioned in their textbooks for his invention of the foil air bearing, still very much in use in turbines. It was like he was a rock star to these students.

I spent a lot of time with Pop, listening to him as he went through his process for finding answers. He questioned and investigated assumptions, including his own. He had a remarkable clarity of mind. It probably didn't hurt that he was a genius.

What I learned from him was that process of investigating and questioning, and it's now automatic. That practice is what had enabled me to turn around a number of troubled productions in the course of my career, steering them away from the iceberg. That said, I'm not nearly as rigorous as he was, and I certainly don't have that massive number of cylinders under the hood that he had. But, the instinct is always active, whether I choose to pursue it or not.

There is an argument, and you hit upon it here, that it is not critical thinking that's important, but clear thinking. Clarity of thought. Critical thinking might be part of the pathway to clarity, but clear unadulterated understanding is the goal.

 

[sablebrush52]

There is an argument, and you hit upon it here, that it is not critical thinking that's important, but clear thinking. Clarity of thought. Critical thinking might be part of the pathway to clarity, but clear unadulterated understanding is the goal.

True, and one is unlikely to get to one without the other.

 

[woodsroad]

Critical thought can be transmitted to others. It's a matter of learning (i.e. training) like anything else that requires some discipline and study to pick up. Not talking about "trained lapdogs" not that you've made yourself clear about what you mean by that term.

I take exception to the notion that there needs to be a gatekeeper of knowledge that bestows the the ability to think critically. While I certainly do hold that education is the key to health, happiness and a free society, I also believe that most of us are born with an innate ability to navigate information and discern the path forward.

My mother, who had no education beyond high school, was one of the best critical thinkers that I've ever met. She was a hospital lab technician and every day, the results of her work could make or break hundreds of people's lives. This was in the pre-computer era, when all her work, although performed on instruments, had to be carefully analyzed and observed for extraneous influences like human error, bad reagents or faulty equipment. There was a lot of troubleshooting involved when controls didn't come out right and doctors and patients were awaiting results. She got her critical thinking skills from her parents, neither of which had any formal education beyond the Talmud and Torah, but were fierce skeptics.

So, while critical thinking skills can (and should) certainly be taught, too often I've seen it wrapped up with ideology. I work in higher ed. Time and time again I've seen professors hold themselves up as the gatekeepers to critical thought, much like the bridge troll in The Holy Grail. And yet, some of the smartest and most able-minded people that I've met, troubleshooters, investigators and reporters, have no formal training in critical thought. Conversely, I've met plenty of people who fancy themselves intellectuals and critical thinkers simply because, like the scarecrow in Oz, they've been told that they are.

 

[Pypkė]

Well that's kind way past what I was thinking of, but thanks for clarifying.

 

[sardonicus87]

Too many people these days:

Harris Alterman on Instagram

2M likes, 21K comments - theharrisalterman on October 6, 2025

www.instagram.com

 

[Pypkė]

We can think our way out of not knowing, but we can't know our way out of not thinking.

 

[jaingorenard]

I take exception to the notion that there needs to be a gatekeeper of knowledge that bestows the the ability to think critically. While I certainly do hold that education is the key to health, happiness and a free society, I also believe that most of us are born with an innate ability to navigate information and discern the path forward.

My mother, who had no education beyond high school, was one of the best critical thinkers that I've ever met. She was a hospital lab technician and every day, the results of her work could make or break hundreds of people's lives. This was in the pre-computer era, when all her work, although performed on instruments, had to be carefully analyzed and observed for extraneous influences like human error, bad reagents or faulty equipment. There was a lot of troubleshooting involved when controls didn't come out right and doctors and patients were awaiting results. She got her critical thinking skills from her parents, neither of which had any formal education beyond the Talmud and Torah, but were fierce skeptics.

So, while critical thinking skills can (and should) certainly be taught, too often I've seen it wrapped up with ideology. I work in higher ed. Time and time again I've seen professors hold themselves up as the gatekeepers to critical thought, much like the bridge troll in The Holy Grail. And yet, some of the smartest and most able-minded people that I've met, troubleshooters, investigators and reporters, have no formal training in critical thought. Conversely, I've met plenty of people who fancy themselves intellectuals and critical thinkers simply because, like the scarecrow in Oz, they've been told that they are.

Have you ever read The Chosen by Potok? It's amazing how studying Torah and Talmud can do exactly what you've described, and make people more skeptical, and really hone critical thinking. Judaism must be one of the few religions to have questioning right at the heart (four questions at Passover, for example, asked by the youngest person present).

 

[woodsroad]

Have you ever read The Chosen by Potok? It's amazing how studying Torah and Talmud can do exactly what you've described, and make people more skeptical, and really hone critical thinking. Judaism must be one of the few religions to have questioning right at the heart (four questions at Passover, for example, asked by the youngest person present).

My wife, raised Lutheran, is still amazed at how Judaism is diametrically opposed to Christianity. Questioning is at root of being Jewish.

 

[Chris T]

Have you ever read The Chosen by Potok? It's amazing how studying Torah and Talmud can do exactly what you've described, and make people more skeptical, and really hone critical thinking.

The Methodist tradition in Protestant Christianity used to have the same emphasis (the "Wesleyan Quadrilateral", in shorthand.) It may still be there with some groups, but sadly, in my own experience (USA), it seems to have been mostly abandoned.

So now I've got The Chosen in my Amazon cart, and a good excuse to open a tin of either Gates of Light or Daylight (C&D Hebraica) when it arrives

 

[Dr. Van Loafer]

Turns out people are putting all kinds of information into AI….no surprise there and also no surprise is that now the “almighty” Facebook is going to sell the data you put into their LLM so they can target you with advertisements. This article concludes with a helpful reminder:

“There is an old adage that if you’re not paying for a service, then you are not the customer, but the product. In this new era of AI, this saying may need to be rewritten to replace the word product with prey.”