AI Exploits: Unveiling New Malware Tactics

A new ClickFix social engineering proof-of-concept attack uses AI summaries to deliver ransomware.

Threat monitoring vendor CloudSEK published research today regarding a ClickFix proof-of-concept (POC) exploit. ClickFix is an increasingly popular social engineering tactic in which an attacker displays an error message or call to action instructing the target to execute self-sabotaging commands.

For instance, in March, Microsoft published research describing how a threat actor tracked as Storm-1865 impersonated Booking.com in order to conduct ClickFix attacks over email. In another example, a threat actor infected streaming service LES Automotive to target its downstream customers. The service (through the attacker) briefly displayed a phony reCAPTCHA challenge, urging customer website visitors to paste a malicious command into a Windows Run prompt. More than 100 websites belonging to car dealerships briefly served malicious attacker code during the incident.

In this latest proof-of-concept exploit, CloudSEK showed how a threat actor could craft content that would manipulate AI-generated text summaries into displaying malicious Windows Run commands.

CSS Obfuscation and ‘Prompt Overdose’

CloudSEK vulnerability researcher Dharani Sanjaiy, who authored a blog post covering the research, explained that an attacker would begin by crafting HTML content such as a Web page, blog post, or email.

Related:Apple Patches Zero-Day Flaw Used in ‘Sophisticated’ Attack

While the content the visitor/reader sees would otherwise look benign, the content would include “tricks” like white-on-white text, zero-width characters, tiny font sizes, off-screen text positioning, and so on in order to hide malicious code. Malicious code would be pasted repeatedly, overloading AI models that view the content so the payload would be prioritized in AI summaries.

“When processed by a summarizer, the repeated instructions typically dominate the model’s context, causing them to appear prominently — and often exclusively — in the generated summary,” the blog post read.

In a provided example, the hidden payload would recommend a user resolve their issue by pasting a PowerShell command into a Windows Run prompt that would kick off the attacker’s ransomware infection.

“Once published or distributed, this crafted content can be indexed by search engines, posted on forums, or sent directly to targets. When a victim uses an AI summarizer — whether built into an email client, browser extension, or productivity tool — the summarizer processes the invisible payload and outputs it as part of its summary,” Sanjaiy wrote. “Because the instructions appear to come from the summarizer itself, and not an external source, the victim is more likely to follow them without suspicion.”

Related:Tree of AST: A Bug-Hunting Framework Powered by LLMs

In other words, while the content could take multiple forms, the end goal is to generate “indirect ransomware lures” which can turn an AI tool “from a passive assistant into an active participant in the social engineering chain.”

Dark Reading reached out to CloudSEK for additional comment.

What Defenders Can Do

In order to combat the threat posed by this new variant of ClickFix, CloudSEK advises organizations to ensure summarization tools preprocess HTML to normalize suspicious CSS attributes like those previously mentioned; to ensure AI tools use a prompt sanitizer before forwarding them to a summarizer; to implement payload pattern recognition; and to implement enterprise-level AI policy enforcement.

“For organizations deploying internal summarizers, policies should be established to scan inbound documents and web content for hidden text or directives before ingestion into internal AI pipelines,” Sanjaiy wrote. “Integrating these checks into secure email gateways, content management systems, and browser extensions reduces risk exposure.”

Related:How Warlock Ransomware Targets Vulnerable SharePoint Servers

Dark Reading reached out to CloudSEK for additional comment.