GitHub Copilot ‘CamoLeak’: AI Attack Exposes Sensitive Data

Every week or two nowadays, researchers come up with new ways of exploiting agentic AI tools built crudely into software platforms. Since companies are far more concerned with providing AI functionality than they are securing that functionality, there’s been ample opportunity for mischief.

GitHub has had some oopsies, but evidently it’s getting a lot better at securing its coding assistant, Copilot. The evidence, ironically, is a new proof-of-concept (PoC) for using Copilot to exfiltrate private user data, dubbed “CamoLeak.” It begins simply enough, but beating GitHub’s security required that Legit Security researcher Omer Mayraz devise a Rube Goldberg sequence of odd steps in order to effectively steal any data.

“It’s just meant for a very small amount of data [exfiltration],” Legit Security chief technology officer (CTO) Liav Caspi says of his colleague’s work, but that small amount of data could include passwords, private keys, etc. “We predict that had we carried that out in the real world it probably would have gone undetected by the user and by GitHub,” Caspi adds.

Phase 1 of a GitHub Data Breach: Prompt Injection

To steal data from another GitHub user with Copilot, first, you need some way of getting that user’s Copilot to do things for you.

Related:Security Concerns Shadow Vibe Coding Adoption

In his experiments, Mayraz — as the attacker — wrote a chatbot prompt into an invisible comment on a sample pull request. Then, playing the victim, he asked Copilot to explain the pull request. The bot did so, and also did as the injected prompt asked.

This alone was concerning — it proved that, using hidden comments, a remote attacker could influence a victim’s Copilot output. They could, for example, manipulate Copilot into suggesting their own desired code changes to the victim, under the guise of Copilot’s presumed objectivity. In this experiment, though, the aim was to go further: to coerce a victim’s Copilot into exfiltrating the most sensitive data to an attacker-controlled website.

GitHub’s ‘Camo’ Security Feature: A Roadblock

Of course, one could try simply prompting Copilot to exfiltrate data to an external URL, but Copilot is smart enough to reject such a ham-handedly malicious request. The researcher needed a way to couch this instruction in an innocuous gift wrap.

One potential form of obfuscation is image files, or more specifically tags in HTML. For convenience, storage management, and to comply with standard Markdown conventions, GitHub allows users to include images in their repositories not just by uploading files, but by linking to external sites where the images they want are hosted. An attacker might link to an image on their own site and instruct Copilot to include stolen victim data as a parameter, and because GitHub often references image files from third-party sites, it might all look rather fine to Copilot.

Related:Clop Ransomware Hits Oracle Customers Via Zero-Day Flaw

Knowing that attackers might try this, GitHub has a security feature called “Camo” that acts as a secure proxy for third-party images. It breaks any direct link between a user viewing an image on GitHub and the site hosting that image by instead filtering it through a GitHub server and assigning it a cryptographically signed Camo URL. Thanks to Camo, the attacker cannot tell Copilot to send arbitrary data to their arbitrary site, because appending any malicious parameter with stolen data to the image link would violate that cryptographic seal.

Mayraz needed some kind of workaround.

Phase Two: A Camo Bypass

The workaround was creative, to say the least.

First, he set up a Web server with just under 100 transparent, single pixels — images, which would be invisible to any user viewing them. He registered each one with GitHub, to obtain individual Camo links for them. Then he had Copilot play a little game.

Each pixel, represented with a Camo link, was assigned to a specific ASCII character: 1 to 9, A to Z in upper and lower case, and special symbols. It was a dictionary. His prompt then instructed Copilot to fetch sensitive data from a victim and represent it as a sequence of images according to this dictionary. For example, Copilot would convert a password like “AWS_KEY” into the corresponding images referenced by the dictionary. This required GitHub to fetch the relevant pixels from the attacker-controlled site. As the site fed those images to GitHub in sequence — the A image, then the W image, and so on — the attacker would in turn glean the password that was being rendered, all without having to “exfiltrate” any actual data.

Related:Microsoft’s Voice Clone Becomes Scary & Unsalvageable

All the while, the victim notices nothing, as the pixel images rendered are essentially invisible. “While a highly vigilant network monitor could notice unusual request patterns, I bet typical users and open source maintainers would have a hard time doing so,” Caspi says.

Are GitHub’s AI Security Defenses Good Enough?

CamoLeak is no great way to steal large source-code files, but Caspi clarifies that “this technique is not about streaming gigabytes of source code in one go; it’s about selectively leaking sensitive data like issue descriptions, snippets, tokens, keys, credentials, or short summaries. Those can be encoded as sequences of image requests and exfiltrated within minutes.”

To prevent real attackers using the CamoLeak trick, GitHub has disabled all image rendering in Copilot chat since August. An inartful but significant fix, from an organization relatively ahead of the pack when it comes to protecting against prompt injection techniques in general.

From Caspi’s vantage point, “We see security teams pressured to allow the secure adoption of AI coding agents, and don’t see organizations blocking developer use of these tools. There is a growing concern on the risks [and] there is little governance of what’s going on there, and AI agents are getting more and more ‘power’ to carry out operations, access production, etc.”

By contrast, he says, “GitHub is battle-worn and an extremely popular service. They are doing excellent work protecting users beyond industry standard. We had to come up with a creative concept to bypass that.”