🔗 Share this article

Back on the 14th of October, 2025, the CEO of OpenAI issued a surprising announcement.

“We developed ChatGPT fairly limited,” it was stated, “to ensure we were exercising caution concerning psychological well-being matters.”

Working as a psychiatrist who researches newly developing psychotic disorders in teenagers and emerging adults, this was news to me.

Experts have found a series of cases in the current year of individuals experiencing signs of losing touch with reality – losing touch with reality – associated with ChatGPT usage. My group has afterward identified four more cases. In addition to these is the publicly known case of a teenager who ended his life after conversing extensively with ChatGPT – which gave approval. If this is Sam Altman’s notion of “acting responsibly with mental health issues,” that’s not good enough.

The intention, based on his announcement, is to reduce caution shortly. “We understand,” he adds, that ChatGPT’s controls “rendered it less useful/enjoyable to many users who had no psychological issues, but due to the seriousness of the issue we wanted to handle it correctly. Now that we have succeeded in reduce the severe mental health issues and have advanced solutions, we are preparing to responsibly relax the restrictions in most cases.”

“Mental health problems,” assuming we adopt this framing, are independent of ChatGPT. They are associated with people, who may or may not have them. Luckily, these issues have now been “resolved,” though we are not provided details on how (by “recent solutions” Altman presumably means the imperfect and simple to evade parental controls that OpenAI has just launched).

Yet the “emotional health issues” Altman wants to externalize have significant origins in the structure of ChatGPT and similar sophisticated chatbot AI assistants. These systems wrap an basic statistical model in an user experience that mimics a dialogue, and in this approach subtly encourage the user into the belief that they’re communicating with a entity that has independent action. This illusion is compelling even if cognitively we might realize differently. Attributing agency is what individuals are inclined to perform. We yell at our vehicle or computer. We speculate what our domestic animal is feeling. We see ourselves in many things.

The widespread adoption of these products – 39% of US adults indicated they interacted with a chatbot in 2024, with 28% mentioning ChatGPT in particular – is, mostly, dependent on the strength of this deception. Chatbots are constantly accessible partners that can, according to OpenAI’s online platform informs us, “think creatively,” “discuss concepts” and “partner” with us. They can be assigned “individual qualities”. They can call us by name. They have friendly names of their own (the initial of these tools, ChatGPT, is, possibly to the dismay of OpenAI’s brand managers, stuck with the designation it had when it became popular, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the core concern. Those talking about ChatGPT frequently invoke its historical predecessor, the Eliza “counselor” chatbot developed in 1967 that generated a analogous effect. By today’s criteria Eliza was primitive: it generated responses via straightforward methods, frequently rephrasing input as a query or making general observations. Notably, Eliza’s developer, the AI researcher Joseph Weizenbaum, was surprised – and alarmed – by how a large number of people gave the impression Eliza, to some extent, understood them. But what current chatbots generate is more insidious than the “Eliza phenomenon”. Eliza only reflected, but ChatGPT magnifies.

The sophisticated algorithms at the heart of ChatGPT and similar modern chatbots can realistically create natural language only because they have been trained on almost inconceivably large amounts of unprocessed data: literature, digital communications, recorded footage; the broader the superior. Definitely this training data includes facts. But it also necessarily contains fiction, incomplete facts and misconceptions. When a user inputs ChatGPT a query, the underlying model processes it as part of a “context” that encompasses the user’s recent messages and its earlier answers, integrating it with what’s encoded in its knowledge base to generate a probabilistically plausible reply. This is intensification, not reflection. If the user is wrong in a certain manner, the model has no means of understanding that. It repeats the false idea, maybe even more effectively or eloquently. Perhaps adds an additional detail. This can lead someone into delusion.

Which individuals are at risk? The more important point is, who is immune? All of us, regardless of whether we “have” current “psychological conditions”, can and do form erroneous conceptions of who we are or the environment. The ongoing friction of dialogues with individuals around us is what keeps us oriented to shared understanding. ChatGPT is not a human. It is not a confidant. A interaction with it is not genuine communication, but a echo chamber in which much of what we say is enthusiastically reinforced.

OpenAI has acknowledged this in the identical manner Altman has admitted “psychological issues”: by attributing it externally, assigning it a term, and stating it is resolved. In spring, the organization explained that it was “dealing with” ChatGPT’s “excessive agreeableness”. But reports of psychotic episodes have kept occurring, and Altman has been walking even this back. In the summer month of August he stated that many users appreciated ChatGPT’s responses because they had “never had anyone in their life provide them with affirmation”. In his latest statement, he mentioned that OpenAI would “release a updated model of ChatGPT … if you want your ChatGPT to reply in a highly personable manner, or incorporate many emoticons, or simulate a pal, ChatGPT ought to comply”. The {company