Legit Security has detailed a vulnerability in the GitHub Copilot Chat AI assistant that led to sensitive data leakage and full control over Copilot’s responses.
Combining a Content Security Policy (CSP) bypass with remote prompt injection, Legit Security’s Omer Mayraz was able to leak AWS keys and zero-day bugs from private repositories, and influence the responses Copilot provided to other users.
Copilot Chat is designed to provide code explanations and suggestions, and allows users to hide content from the rendered Markdown, using HTML comments.
A hidden comment would still trigger the usual pull request notification to the repository owner, but without displaying the content of the comment. However, the prompt is injected into other users’ context as well.
The hidden comments feature, Mayraz explains, allows a user to influence Copilot into displaying code suggestions to other users, including malicious packages.
Mayraz also discovered that he could craft prompts containing instructions to access users’ private repositories, encode their content, and append it to a URL.
“Then, when the user clicks the URL, the data is exfiltrated back to us,” he notes.
However, GitHub’s restrictive CSP blocks the fetching of images and other content from domains not owned by the platform, thus preventing data leakage by injecting an HTML tag into the victim’s chat.
When external images are included in a README or Markdown file, GitHub parses them to identify the URLs, and generates an anonymous URL proxy for each file using the open source project Camo.
The external URL is rewritten to a Camo proxy URL and, when the browser requests the image, the Camo proxy checks the URL signature and fetches the external image from the original location only if the URL was signed by GitHub.
This prevents the exfiltration of data using arbitrary URLs, ensures security by using a controlled proxy to fetch images, and does not expose the image URL when it is displayed in the README.
“Every tag we inject into the victim’s chat must include a valid Camo URL signature that was pre-generated. Otherwise, GitHub’s reverse proxy won’t fetch the content,” Mayraz notes.
To bypass the protection, the researcher created a dictionary of all letters and symbols in the alphabet, pre-generated corresponding Camo URLs for each of them, and embedded the dictionary into the injected prompt.
He created a web server that responded with a 1×1 transparent pixel to each request, created a Camo URL dictionary of all the letters and symbols he could use to leak sensitive content from repositories, and then built the prompt to trigger the vulnerability.
Mayraz has published proof-of-concept (PoC) videos demonstrating how the attack could be used to exfiltrate zero-days and AWS keys from private repositories.
On August 14, GitHub notified the researcher that the issue had been addressed by disallowing the use of Camo to leak sensitive user information.
Related: Critical Vulnerability Puts 60,000 Redis Servers at Risk of Exploitation
Related: Microsoft and Steam Take Action as Unity Vulnerability Puts Games at Risk
Related: GitHub Boosting Security in Response to NPM Supply Chain Attacks
Related: Code Execution Vulnerability Patched in GitHub Enterprise Server
