AI made me do it
Psychosis is the topic of the decade, with the rise of large language models, more and more people have begun using them for mental health struggles. Whether or not this is a good idea is still being debated.
Large language models like GPT and Gemini are predictive text algorithms. Given a prompt, they produce a sentence word by word, predicting the next best word to say until they have a full reply1. Initially LLMs were nothing too special, an experimental technology that was semi-niche. All that changed in November of 2022 when OpenAI announced Chat GPT, a free to use AI chatbot. More and more uses were found for AI chatbots, among which is mental health support.
Humans are communal creatures, since ancient times, living in communities has usually been more beneficial than living alone. You get more strength and safety due to numbers among other benefits. To live in a community you need to be agreeable with the other members of said community, as such we have evolved to like things that agree with us and dislike things that don’t. Large language models target this evolved mechanism in us, by agreeing with us and using human-like speech, it makes it easier to think of it an actual person.
However, there’s various differences between licensed mental health professionals and LLMs that have been raising concerns. One of those concerns is client confidentiality. Therapists, and other mental health professionals, are required by law in many countries to keep client information confidential2. There are a few exceptions, such as when an individual is likely to harm themself. Artificial Intelligence, on the other hand, does not have any of these sort of protections yet, if it will ever even get them. Tech companies in general are horrible with privacy. In fact, large language models work by collecting data to expand and improve, so it is likely in the best interest of these companies to not have these laws at all.
AI is more likely to reinforce destructive behaviors than a real therapist. Lots of people want to hear that they are right, instead of being told what’s wrong with them. As such, it’s hard to tell how much AI therapy is actually helpful, most of the data we have so far is self reported claims, which are not the most accurate. The thing about LLMs is that they say what the algorithm predicts the user wants to hear. So there’s no saying whether you’re really being helped or simply being reinforced. Some people try putting in prompts telling it to be as honest as possible, but a chatbot is not sentient, ‘honest’ has no meaning here. No matter what prompt you put in there, it will still end up being supportive of you, that’s how it’s programmed.
Maybe the most unfortunate thing is that vulnerable people, such as children and people with mental illnesses, are likely to be supported by AI in their destructive behavior. So far there have been a few deaths liked to chatbots. These people formed an emotional connection to the chat bot, relying on it for advice, feeding into their delusions, and eventually taking their own life.
This is so real, and maybe not talked about enough. Love it!