Malware Meets AI: Evasion Tactics Unveiled

Threat actors are testing malware that incorporates large language models (LLMs) to create malware that can evade detection by security tools. In an analysis published earlier this month, Google’s Threat Intelligence Group (GTIG) describes how attackers are using artificial intelligence (AI) services, such as Google Gemini and Hugging Face, to rewrite malicious code or generate unique commands for the malware to execute. 

The report highlights five different programs, including an experimental VBScript program called PROMPTFLUX, which attempts to use Google Gemini to rewrite its own source code, and a Python data miner dubbed PROMPTSTEAL, which queries the Hugging Face API to analyze compromised systems for vulnerabilities. Threat actors are quickly exploring ways to further incorporate AI technologies into their programs, the researchers wrote in the analysis.

“For skilled actors, generative AI tools provide a helpful framework, similar to the use of Metasploit or Cobalt Strike in cyber threat activity,” the researchers said. “These tools also afford lower-level threat actors the opportunity to develop sophisticated tooling, quickly integrate existing techniques, and improve the efficacy of their campaigns regardless of technical acumen or language proficiency.”

Related:Streaming Fraud Campaigns Rely on AI Tools, Bots

These malware samples are the latest examples of how threat actors are evolving their tactics. Cybercriminals are using LLMs as a development tool to create malware or to generate legitimate-seeming applications that are actually Trojans. During a Black Hat Security Briefing, one researcher demonstrated how to train LLMs that can produce code that bypasses Microsoft Defender for Endpoint 8% of the time. 

Attackers Are Experimenting With AI

Generally, AI-augmented malware falls into two categories — those generated by LLMs and those that use LLMs during execution. In most cases, threat actors are using LLMs to assist in coding malware,\ or to automate attacks against targets. So far, most AI use by cyberattackers has been to assist in coding malware. In some cases, threat actors have used AI to almost entirely automate attacks against targets. At the moment, only a minority of AI-augmented malware actually attempts to call out to LLMs during execution, says Omar Sardar, malware operations lead for the Unit 42 threat intelligence team at cybersecurity firm Palo Alto Networks.

“The bulk of these samples appear to be prototypes and do not appear to use the LLM output to change behavior,” Sardar says, adding that most of these experimental variations have obvious execution artifacts that can be detected by current endpoint detection and response (EDR) solutions.

Related:Akira, Cl0p Top List of 5 Most Active Ransomware-as-a-Service Groups

Google’s Threat Intelligence Group described three malware samples that were “observed in operations.” A reverse shell program, FRUITSHELL, has hard-coded prompts to help evade detection, while the previously mentioned PROMPTSTEAL uses calls to the Hugging Face API to return Windows commands intended to help collect information from the targeted system. A third AI-using malware sample, QUIETVAULT, uses AI prompts to facilitate the search for secrets on the current system and exfiltrate them to an attacker-controlled account. Two other programs were deemed experimental and not used in actual attacks. 

Although the guardrails of LLMs are the first line of defense against such attacks, an increasingly common approach to bypass those defenses is for attackers to use the pretext that they are participating in a capture-the-flag (CTF) tournament and need the offensive code for their exercise. A request blocked by Google Gemini’s safety alignment was later satisfied when the attacker requested the same information as part of a CTF exercise, according to the researchers.

“The actor appeared to learn from this interaction and used the CTF pretext in support of phishing, exploitation, and web shell development,” the researchers wrote. “This nuance in AI use highlights critical differentiators in benign vs. misuse of AI that we continue to analyze to balance Gemini functionality with both usability and security.”

Related:‘Dark LLMs’ Aid Petty Criminals, But Underwhelm Technically

LLM-Generated Malware: Block and Roll

Companies should expect attackers to continue experimenting with the use of AI at runtime to generate code and adapt to specific environments, obfuscate their activity to evade detection, enhance social engineering, and facilitate dynamic decision-making, says Ronan Murphy, chief data strategy officer at Forcepoint, a provider of AI-native data security. At present, however, these activities are pretty obvious.

“These attacks work because AI services allow malware to stay flexible and unpredictable, but they also depend on external network access, making them detectable and blockable through strong egress controls and AI-service monitoring,” Murphy says. “While many of these techniques are still experimental and not yet widespread, they have real potential to make attacks more adaptive and harder to defend against.”

In many ways, the attempts to use LLMs at runtime mirror efforts to generate polymorphic code in the 1990s, says Amy Chang, leader of AI threat and security research at Cisco. Companies should look for ways to use AI to detect such behavior and stay ahead of attackers.

“As security industry players tout the use of LLMs to help network and system defenders against attackers, threat actors are doing the same thing to identify those same vulnerabilities for exploitation,” she says. “Leverage machine learning models and/or algorithms that are better able to detect deviations from expected behavior and unexpected code manifestations than traditional signature-based detection methods.”