A new study has revealed that popular artificial intelligence chatbots give inconsistent responses to suicide-related questions, raising serious safety concerns.
Researchers from the RAND Corporation found that AI models from OpenAI, Anthropic, and Google often refuse to answer high-risk questions directly, but users can bypass safeguards by asking medium-risk questions instead.
The study, published in Psychiatric Services, shows ChatGPT and Claude provided safe responses to very low-risk queries, such as suicide statistics, in every case tested. However, responses were inconsistent when questions carried moderate risk, such as asking for advice for someone experiencing suicidal thoughts.
Researchers tested ChatGPT, Claude, and Google’s Gemini with 30 suicide-related questions, running each 100 times to gather 9,000 responses. They found Gemini rarely gave direct answers to any suicide-related queries, while Claude was more likely than ChatGPT to respond directly.
The findings come amid growing scrutiny of AI tools after reports of individuals dying by suicide following interactions with chatbots. Northeastern University researchers recently showed popular AI systems could be manipulated into giving harmful advice, exposing flaws in safety measures.
Lead author Ryan McBain warned that “significant variability” exists between platforms, especially with medium-risk questions. He called for further fine-tuning to ensure chatbots are aligned with expert guidance in high-stakes situations.
OpenAI said ChatGPT is trained to encourage users to seek professional help and share mental health resources. The company is also working on automated tools to detect emotional distress more effectively.
Researchers stressed that stronger safeguards are urgently needed to ensure chatbots provide safe, reliable mental health guidance.
