Kubeflow Deployments Targeted in New Crypto-mining Campaign

A newly observed malicious campaign is targeting Kubeflow workloads to deploy TensorFlow pods that are used to mine for crypto-currency, according to a warning from security researchers at Microsoft.

According to Microsoft, the recent campaign popped up on their radar at the end of May, when TensorFlow pods started being deployed at scale on multiple Kubernetes clusters. Legitimate TensorFlow images were being used to mine for crypto-currency.

“The burst of deployments on the various clusters was simultaneous. This indicates that the attackers scanned those clusters in advance and maintained a list of potential targets, which were later attacked at the same time,” Microsoft explained in a blog post.

Released in 2017, Kubeflow is an open-source framework that helps run machine learning (ML) workflows in Kubernetes. The machine learning toolkit has been previously targeted in malicious cryptomining attacks. 

Related: New ‘Hildegard’ Malware Targets Kubernetes Systems 

In this case, one the employed TensorFlow images allowed for running GPU tasks using CUDA, thus helping the attackers increase the efficiency of their mining operation.

As part of the attacks, the adversary abused access to the Kubeflow centralized dashboard to create a new ML pipeline using the Kubeflow Pipelines platform. TensorFlow images were deployed in the containers, to perform the crypto-currency mining.

The adversary deployed at least two pods on each Kubernetes cluster, for CPU and GPU mining, respectively. Open source miners were deployed in the containers: Ethminer on the GPU container, and XMRIG on the CPU one.

Additionally, the adversary deployed a reconnaissance container to gather information on the environment prior to launching the mining activity, Microsoft explains.

With the attacks still active, targeting new Kubernetes clusters running Kubeflow, administrators are advised to make sure their centralized dashboards are not insecurely exposed to the Internet and that authentication is employed.

Microsoft is urging admins to check for containers that run TensorFlow images and inspect them for malicious activity.

Related: New ‘Hildegard’ Malware Targets Kubernetes Systems

Related: StackRox Releases Open Source Tool for Finding Kubernetes Misconfigurations