OpenAI Admits a Core AI Security Flaw Is 'Unlikely to Ever Be Fully Solved'

OpenAI has conceded that one of the most critical vulnerabilities in modern AI systems, known as <keyword>prompt injection</keyword>, is a risk that’s not going away anytime soon. In a Monday blog post, the company stated that these attacks—which manipulate AI agents into following malicious instructions hidden in web pages or emails—are "unlikely to ever be fully ‘solved,’” comparing them to the persistent nature of scams and social engineering. This admission raises urgent questions about how safely autonomous AI agents can truly operate on the open web.

A Widely Recognized, Unsolvable Problem

OpenAI isn't alone in this grim assessment. The U.K.’s National Cyber Security Centre (NCSC) warned earlier this month that <keyword>prompt injection</keyword> attacks "may never be totally mitigated." The government agency advised cyber professionals to focus on reducing the risk and impact rather than thinking the attacks can be "stopped." For its part, OpenAI views the issue as a "long-term AI security challenge," acknowledging that its "agent mode" in <keyword>ChatGPT Atlas</keyword> significantly "expands the security threat surface."

Fighting AI with AI: The Automated Attacker

Sponsored

Advertise with Us

[email protected]

Sponsored

The company’s answer to this Sisyphean task is to build its own highly advanced hacker: an "LLM-based automated attacker." This bot, trained using reinforcement learning, is designed to proactively discover novel attack strategies before they are exploited in the wild. According to OpenAI, this automated attacker has already uncovered sophisticated, long-horizon attack methods that were missed by human red teaming campaigns.

Its key advantage is its insider access. The bot can see the target AI’s internal reasoning, allowing it to rapidly test an attack, study the response, tweak the exploit, and try again. While rivals like Google and Anthropic also emphasize layered defenses and continuous testing, OpenAI's internal AI attacker represents a more aggressive, self-testing approach to hardening its systems.

Expert Reality Check: Does the Value Justify the Risk?

Despite these efforts, security experts remain cautious. Rami McCarthy, principal security researcher at Wiz, told TechCrunch that the risk in AI systems can be understood as "autonomy multiplied by access." Agentic browsers like <keyword>ChatGPT Atlas</keyword> are in a particularly dangerous position, combining moderate autonomy with extremely high access to sensitive data like emails and payment information.

"For most everyday use cases, agentic browsers don’t yet deliver enough value to justify their current risk profile," McCarthy said, suggesting the trade-offs are still very real. OpenAI itself recommends users constrain their agents by giving them specific instructions and requiring user confirmation before sending messages or making payments—an implicit acknowledgment of the current limitations.