Connect with us

Hi, what are you looking for?

SecurityWeekSecurityWeek

Application Security

New Class of CI/CD Attacks Could Have Led to PyTorch Supply Chain Compromise

Researchers detail a CI/CD attack leading to PyTorch releases compromise via GitHub Actions self-hosted runners.

Software Supply Chain Attack

A newly disclosed class of CI/CD attacks could have allowed attackers to inject malicious code into the PyTorch repository, leading to massive supply chain compromise, Praetorian security researcher John Stawinski says.

Initially detailed in December 2023, the attack method targets GitHub repositories with self-hosted runners attached and allows a threat actor to execute arbitrary code without requiring approval.

In short, an attacker can use a fork pull request to become a contributor to a repository that has a self-hosted runner attached, and then be able to run any GitHub workflow on the runner. If the runner was configured using the default steps, it is non-ephemeral, enabling persistent access.

The attack flow was discovered by Adnan Khan, who used it against GitHub’s own actions/runner-images repository and received a $20,000 bug bounty reward. Khan and Stawinski then identified thousands of other GitHub repositories prone to the attack.

The machine learning (ML) framework PyTorch, Stawinski explains, was one of their first targets, given its popularity. The child of Meta AI and now part of the Linux Foundation, PyTorch is used in various popular deep learning models.

Following the same steps that allowed them to gain access to GitHub’s repository, the researchers discovered that PyTorch used self-hosted runners that did not require workflow approval for fork pull requests from previous contributors, which allowed them to mount their attack.  

Once they gained access, the researchers installed their own self-hosted runner on the vulnerable PyTorch runner, which allowed them to maintain persistence without raising suspicion.

However, they were more interested in the post-exploitation activities they could perform, hoping that a broad level of access would draw attention to the attack and trigger a prompt response.

Advertisement. Scroll to continue reading.

The researchers were able to extract GitHub secrets used by PyTorch, including several sets of AWS secret access keys and GitHub Personal Access Tokens (PATs) that could allow them to perform various operations.

“Our exploit path resulted in the ability to upload malicious PyTorch releases to GitHub, upload releases to AWS, potentially add code to the main repository branch, backdoor PyTorch dependencies – the list goes on,” Stawinski says.

The researchers discovered they could trigger a workflow that used the compromised GitHub PATs to authenticate to the code hosting platform, and that those secrets “had access to over 93 repositories within the PyTorch organization, including many private repos, and administrative access over several”.

Using these compromised secrets, an attacker could modify releases, add code directly to the PyTorch main branch, or set up other paths to supply chain compromise.

“If the threat actor wanted to be more stealthy, they could add their malicious code to one of the other private or public repositories used by PyTorch within the PyTorch organization. Or they could smuggle their code into a feature branch, or steal more secrets, or do any number of creative techniques to compromise the PyTorch supply chain,” Stawinski notes.

In August 2023, the researchers submitted a vulnerability report to Meta, which informed them two months later that the issue was considered mitigated. After more back-and-forth messages discussing remediation, Meta said it issued a $5,000 bug bounty reward for the finding.

The mitigations for this attack are the same that apply to the GitHub Actions chain: the use of isolated, ephemeral self-hosted runners, and requiring approval for all pull requests coming from outside contributors.

“The issues surrounding these attack paths are not unique to PyTorch. They’re not unique to ML repositories or even to GitHub. Threat actors are starting to catch on, as shown by the year-over-year increase in supply chain attacks,” Stawinski concludes.

Related: Major IT, Crypto Firms Exposed to Supply Chain Compromise via New Class of CI/CD Attack

Related: Critical TorchServe Flaws Could Expose AI Infrastructure of Major Companies

Related: Malware Delivered to PyTorch Users in Supply Chain Attack

Written By

Ionut Arghire is an international correspondent for SecurityWeek.

Trending

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Gain valuable insights from industry professionals who will help guide you through the intricacies of industrial cybersecurity.

Register

Join us for an in depth exploration of the critical nature of software and vendor supply chain security issues with a focus on understanding how attacks against identity infrastructure come with major cascading effects.

Register

Expert Insights

Related Content

Application Security

Cycode, a startup that provides solutions for protecting software source code, emerged from stealth mode on Tuesday with $4.6 million in seed funding.

Risk Management

The supply chain threat is directly linked to attack surface management, but the supply chain must be known and understood before it can be...

Cybersecurity Funding

2022 Cybersecurity Year in Review: Top news headlines and trends that impacted the security ecosystem

ICS/OT

The overall effect of current global geopolitical conditions is that nation states have a greater incentive to target the ICS/OT of critical industries, while...

Application Security

Virtualization technology giant VMware on Tuesday shipped urgent updates to fix a trio of security problems in multiple software products, including a virtual machine...

Application Security

Fortinet on Monday issued an emergency patch to cover a severe vulnerability in its FortiOS SSL-VPN product, warning that hackers have already exploited the...

Application Security

GitHub this week announced the revocation of three certificates used for the GitHub Desktop and Atom applications.