Navigating the Rising Tide of Prompt Injections in ChatGPT Atlas

As a new AI-powered Web browser brings agentics closer to the masses, questions remain regarding whether prompt injections, the signature LLM attack type, could get even worse.

ChatGPT Atlas is OpenAI’s large language model (LLM)-powered Web browser launched Oct. 21 and based on Chromium. Currently available for macOS (with other platforms to come), Atlas comes with native ChatGPT functionality including text generation, Web page summarization, and agent capabilities.

OpenAI advertises the agent as being able to “book appointments, create slideshows, and more, handling complex tasks from start to finish.” ChatGPT’s agentic capabilities are only available in Plus (for $20 per month) and Pro ($200 per month), though that is a fair bit more accessible than many of the far more premium agents seen earlier this year. And it’s not alone. A quick search on Google shows a range of similar agentic browsers and extensions at various price levels.

But here’s where things start to get dicey with AI and LLMs. Prompt injections refer to the practice of using a natural language prompt to get an LLM, such as a chatbot, to do something otherwise not intended by the entity responsible for it.

Prompt injections also exist in two forms: direct and indirect. A direct prompt injection, for example, might be to ask a chatbot a question that gets it to divulge sensitive company documentation. An indirect prompt injection is more complex because it involves the attacker inserting a prompt in a situation that does not directly instruct the LLM. This could mean the attacker sends the target an email with a malicious prompt hidden inside the body that an AI assistant reads and follows, or it could mean including a malicious prompt as a hidden element on a Web page that an agent could inadvertently take in as it works.

Related:Infamous Shai-hulud Worm Resurfaces From the Depths

AI vendors have made progress over the years to curb the prompt injection problem, in part by stacking guardrails on top of models to make them less trusting. But with agents, an emerging category of LLM tools that can autonomously use tools and complete tasks, the issue gets so much more complex. 

Agents can do rudimentary coding, analysis, research, security, and other kinds of tasks, including those that require it to work with other agents. When you take LLMs, which already have a history of leaking sensitive data, and give them access to tools, it opens up an organization to an immense attack vector for prompt injections. 

OWASP’s list of agentic AI threats is startling, as prompt injections enable models to use coding tools to create new vulnerabilities, conduct remote code execution attacks, and compromise entire networks of agents. While these might not be the types of attacks we see threat actors exploit every day, agents are an emerging LLM category (a technology which in itself remains nascent).

Related:LINE Messaging Bugs Open Asian Users to Cyber Espionage

Agents were first presented in security as an experimental tool to assist (or perhaps replace) SOC staff and could cost as much as an engineer’s salary. Now, through products like the Atlas browser, the technology could go much wider, therefore opening up the potential for more agentic prompt injections. 

Opening Up the Atlas

In late October, Web browser security firm LayerX reported what it described as the first Atlas browser vulnerability, which would have enabled an attacker to inject malicious instructions into the browser’s memory. The firm tells Dark Reading it is working on additional research involving the browser for publication in the future.

“We need to pay bigger attention to how AI is getting embedded into the browser,” Or Eshed, LayerX cofounder and CEO, explains. “The big problem is what we’re going to see in the next half of a year, where these browsers become more and more powerful. Whatever makes these products more successful is also what will make attackers happier. It’s a kind of double-edged sword.”

Related:The AI Attack Surface: How Agents Raise the Cyber Stakes

The day after Atlas launched, OpenAI chief information security officer (CISO) Dane Stuckey published a post to X both celebrating the launch while noting that prompt injection “remains a frontier, unsolved security problem, and our adversaries will spend significant time and resources to find ways to make ChatGPT agent fall for these attacks.” 

This note was presented alongside a commitment to security, privacy, and safety as well as a list of relevant new features, but it must be said that if OpenAI hasn’t cracked prompt injections yet — for the billions of dollars funneled into it — almost certainly no one else has either. And that’s before one considers how shared responsibility for AI security is kind of a mess.

Amit Chita, field chief technology officer (CTO) for application security vendor Mend.io, tells Dark Reading that even if prompt injection has gotten better in some respects, with agents it’s not so simple. Every tool an agent has access to — and every interaction between tools — represents an additional vector for a prompt injection to exploit. Moreover, agents can’t be held accountable the way human staff can.

“[Agentic AI] just makes the problem more complex, because every tool can take actions that cause data leakage or just harm to the organization,” he says. “The more tools you have, the more opportunity for issues that you have.”

Suresh Batchu, COO and co-founder of browser security vendor Seraphic Security, tells Dark Reading that he expects wider agent availability to make prompt injections “worse in the near-to-medium term.”

“As agents gain autonomy and tool access, prompt injection shifts from ‘make the model say something weird’ to ‘make the model do something dangerous,'” he says. “Cloud providers are already warning that agent toolchains enable exfiltration and remote code execution (RCE) when indirect injections land. More agents mean more targets, more varied implementations, and a long tail of smaller orgs deploying them without mature security. Long term, pressure from repeated incidents will improve defaults, but we’re not there yet.”

Protect Your Agents, Atlas or Otherwise

Let’s say you’re either running an organization and want to try agentic AI or, possibly, a CISO tasked with implementing an agent. 

Chita advises organizations to conduct regular manual reviews to determine what tools and data the agent can access and take an inventory to ensure the agent only has access to what it needs. As he puts it, sometimes an organization may determine there is risk in doing something and are still willing to take the risk, “but you need to do it thoughtfully.”

Batchu recommends strict least-privilege tool access, executing tools in a locked sandbox, placing guardrails at every hop rather than user input and output, and including a human in the loop for high-risk actions. 

“Prompt injections aren’t going away through ‘better prompts,'” he says. “The issue improves when agents are architecturally constrained, tool use is least privileged and sandboxed, and untrusted content is treated as hostile by default.”