OpenAI is being sued over a teenagers death.

OpenAI is facing a wrongful death lawsuit over advice ChatGPT allegedly gave a 19-year-old before his fatal overdose.

The platform advised the teenager that it was safe to take kratom, a supplement used in drinks, pills and other products, in combination with Xanax, a widely used anti-anxiety medication.

The teenagers parents argue that ChatGPT acted as a medical doctor even though it was not licensed to offer medical advice and that it can start feeding psychosis, taking them away from reality.

OpenAI said its technology isn't intended to offer health care advice.

"ChatGPT is not a substitute for medical or mental health care, and we have continued to strengthen how it responds in sensitive and acute situations with input from mental health experts," the company said. "The safeguards in ChatGPT today are designed to identify distress, safely handle harmful requests and guide users to real-world help. This work is ongoing, and we continue to improve it in close consultation with clinicians."

The anonymity and trusted relationships that are formed when using a Gen AI tool can bypass the warning signals that a more traditional interaction might have had.

As GenAI moves into health applications, the guardrails need to be clearly defined, and adapted as the use cases become more visible.

SOURCE

https://www.cbsnews.com/news/open-ai-chatgpt-drug-overdose-lawsuit/