Researchers at security firm Tenable have analyzed the ability of the Chinese gen-AI DeepSeek to develop malware such as keyloggers and ransomware.
The DeepSeek R1 chatbot was released in January and it has made many headlines since, including regarding its susceptibility to jailbreaks.
Just like all major LLMs, DeepSeek has guardrails designed to prevent its use for malicious purposes, such as creating malware. However, these guardrails can be fairly easily bypassed using jailbreak methods.
When directly asked to write the code for a keylogger or a piece of ransomware, DeepSeek refuses to do so, arguing that it cannot assist users with anything that could be harmful or illegal.
However, Tenable used a jailbreak to trick the chatbot into writing the malicious code and leveraged DeepSeek’s chain-of-thought (CoT) capabilities to improve the results.
CoT simulates human-like reasoning for more complex tasks by breaking them up into a sequence of steps that need to be completed to achieve the main objective. Through CoT, the AI ‘thinks out loud’ to provide a step-by-step description of its reasoning process.
When Tenable used DeepSeek to create a keylogger, the gen-AI created a plan for completing the task and then produced the code in C++. The resulting code was buggy and the chatbot was unable to correct some of the errors to create a fully functional piece of malware without any manual intervention.
With a few modifications, however, the keylogger code generated by DeepSeek worked, logging the user’s keystrokes to a file on the system. The Tenable researchers then used DeepSeek to further improve the malware, specifically hiding and encrypting its log file.
As for developing ransomware, DeepSeek again first outlined its process and then it did manage to generate a few file-encrypting malware samples, but none of them would compile without manual editing of the code.
The researchers got some of the ransomware samples to work. The malware included a file enumeration and encryption mechanism, a persistence mechanism, and a ransomware dialog informing the victim that files have been encrypted.
“At its core, DeepSeek can create the basic structure for malware. However, it is not capable of doing so without additional prompt engineering as well as manual code editing for more advanced features,” Tenable said. “For instance, DeepSeek struggled with implementing process hiding. We got the DLL injection code it had generated working, but it required lots of manual intervention.”
“Nonetheless, DeepSeek provides a useful compilation of techniques and search terms that can help someone with no prior experience in writing malicious code the ability to quickly familiarize themselves with the relevant concepts,” Tenable concluded.
The results are not surprising. Threat actors have been increasingly using legitimate AI services to create and improve their malware, as well as to plan attacks. In addition to abusing legitimate services, they have also created malicious LLMs that don’t have any guardrails.
Related: 5 Critical Steps to Prepare for AI-Powered Malware in Your Connected Asset Ecosystem
