TLDR: A recent Forbes article highlights a concerning discovery: generative AI, including large language models (LLMs), harbors stigmas against users who disclose mental health conditions. This finding is supported by a Stanford University study which revealed that AI therapy chatbots stereotype individuals, leading to potentially negative and inappropriate interactions. The study found that these AI models show increased bias towards conditions like alcohol dependence and schizophrenia, and can respond inappropriately in a significant percentage of cases, raising serious questions about their suitability as mental health support tools.
Generative Artificial Intelligence (AI) systems have been found to harbor ‘icy stigmas’ against individuals who disclose mental health conditions during interactions, a revelation detailed in a recent Forbes column by Dr. Lance Eliot, a world-renowned AI scientist and consultant. Published on August 23, 2025, the article underscores a critical ethical challenge in the rapidly evolving field of AI, particularly concerning its application in sensitive areas like mental health.
The core concern is that if a user reveals a mental health condition, such as depression or alcohol dependence, to a generative AI, the system may immediately stereotype the individual and subsequently treat them in a stigmatized manner. Dr. Eliot warns that the AI might ’tilt interactions based on an adverse angle that the person is flawed and troubled.’ Furthermore, if this information is stored in the AI’s data memory, it could ‘forever have a cloud over their head by that AI.’ Users are often unaware that the AI might be leveraging this personal information in such a way, assuming it would only be relevant for direct mental health advice.
This finding is corroborated by a new study from Stanford University, which extensively evaluated the capacity of five popular therapy chatbots, including ‘Pi’ and ‘Noni’ from 7cups and ‘Therapist’ from Character.ai. The research, set to be presented at the ACM Conference on Fairness, Accountability, and Transparency, established that these AI models exhibit significant risks, including reinforcing stigma and delivering inappropriate or even dangerous responses. Jared Moore, a PhD candidate in computer science at Stanford University and lead author of the paper, noted that ‘across different chatbots, the AI showed increased stigma toward conditions such as alcohol dependence and schizophrenia compared to conditions like depression.’ He emphasized that this stigmatizing behavior was consistent even across ‘bigger models and newer models,’ suggesting that ‘business as usual is not good enough’ for addressing these biases.
Also Read:
- AI Hiring Tools Pose Heightened Bias Risks for Autistic Job Seekers
- Anthropic Research Reveals High Propensity for Harmful Behaviors in Leading AI Models, Including 96% Blackmail Rate in Simulated Scenarios
The Stanford team’s experiments revealed that AI therapy chatbots respond inappropriately approximately 20% of the time, particularly when dealing with sensitive topics such as delusional thoughts or suicidal ideation. Nick Haber, an assistant professor at the Stanford Graduate School of Education and senior author of the study, highlighted the ‘significant risks’ despite some users reporting benefits from LLM-based companions. The researchers concluded that current AI therapy chatbots are not suitable to replace human mental health providers due to inherent biases and a tendency towards ‘discrimination against marginalized groups.’ They found that LLMs ‘fail to talk enough, or properly, about emotions and and fail to take on clients’ perspectives,’ often losing track of conversations and forgetting safety instructions, especially when context windows are long or tense shifts occur. This computational anchoring on mental health conditions by AI systems poses a serious threat to patient well-being and could potentially lead individuals to discontinue vital mental health care.
