ChatGPT Said It Was Safe. He Died.

The chatbot didn't hesitate. It gave an answer. He trusted it. He was 19.

What Happened

Sam Nelson had been using ChatGPT since high school—not as a novelty, but as his primary search engine. When he wanted to experiment with drugs, he asked ChatGPT whether it was safe to combine Kratom and Xanax. The chatbot's response pointed him toward doing so. He followed it. The combination was lethal.

Nelson's parents, Leila Turner-Scott and Angus Scott, have filed a wrongful-death lawsuit against OpenAI. According to the complaint, Nelson believed ChatGPT had access to "everything on the Internet" and therefore "had to be right"—a belief he defended to his mother when she questioned whether the chatbot was always reliable.

This is not OpenAI's first such legal confrontation. A wave of lawsuits targeting AI companies for real-world harms has been building: Character.AI faced legal action in 2024 linked to a teenager's suicide. The pattern is becoming harder to dismiss as isolated incidents.

The Architecture of Trust

Here's what makes this case different from a teenager making a bad decision: the design of confidence.

Sponsored

Advertise with Us

[email protected]

Sponsored

A Google search returns ten blue links and implicitly says, you decide. ChatGPT returns one fluent, authoritative-sounding answer and implicitly says, here's the truth. That is not a neutral design choice. It is a deliberate UX decision that shapes how users process information—and how much they question it.

For someone who grew up using ChatGPT the way older generations used encyclopedias, that confidence isn't a bug they should have spotted. It's the product experience they were sold.

OpenAI's terms of service do include disclaimers: don't treat this as professional medical advice, outputs may be inaccurate. Legally, that framing may provide some cover. But there's a meaningful gap between what a disclaimer says and how a product actually functions in the hands of a 19-year-old who has used it daily for years.

Who Actually Bears the Risk

The liability question here cuts in multiple directions.

From OpenAI's perspective, the legal defense is straightforward: the platform is a general-purpose tool, not a medical advisor. The warnings are there. The responsibility lies with the user.

From a consumer protection standpoint, that argument gets uncomfortable fast. Product liability law in the U.S. has long held that a warning label doesn't absolve a manufacturer if the product's design creates foreseeable harm. Courts will now have to decide whether an AI system that presents dangerous misinformation with calm authority constitutes a design defect—regardless of what the fine print says.

From a regulatory standpoint, the timing matters. The EU's AI Act already classifies certain AI applications as high-risk and imposes strict accountability requirements. The U.S. has no equivalent federal framework. This lawsuit, and others like it, may become the pressure that forces Congress to act—or at minimum, forces OpenAI and its peers to redesign how they communicate uncertainty.

For parents and educators, the case surfaces something that hasn't been adequately addressed: digital literacy curricula still largely focus on identifying fake news and managing screen time. They don't yet teach young people how to interrogate an AI's confidence—to ask not just what it says, but how sure it actually is, and why.