I think it's something very similar to the Tesla "autopilot" thing where people believe it more capable than it is. You see so many stories of Tesla wrecks from people engaging autopilot thinking the car can totally drive itself without needing to be monitored, then they take a nap or start dicking around on their phone, because people are lazy and stupid.
I've seen people before on threads about AI who claim they work in AI (which I doubt given what they claim) and that "it definitely is capable of logical thinking", which any reliable article will tell you they, particularly the LLMs (your ChatGPT and Gemini's and whatnot), absolutely are not capable of logic or thinking. They'll tell you exactly how they work... through math and tokens, different tokens have different values and a given token may have a higher or lower value based on context, and all it's doing is being basically the same thing as your phone keyboard's predictive text on steroids.
I agree, often times it's more like... AI is a screwdriver and people are trying to use it like a chisel. That may not be the best analogy, but it's the first I could think of.
My big concerns aren't so much what it is capable of or not, it's more that people will get even more brain-rotted than they currently are. In other words the problem isn't the technology, it's people and how they use it.
And that's not to say there's not other issues on the AI side, like theft of intellectual property and the companies that make them, or their resource hogging, etc.
"
