When AI Becomes a Hacker's Best Friend

$25 Million Gone in One Video Call

In August 2024, cybersecurity researcher Anton Cherepanov spotted something that would shake the security world. What looked like an ordinary file uploaded to VirusTotal turned out to be anything but ordinary. This ransomware could tap into large language models to generate custom code in real-time, map computer systems autonomously, and write personalized ransom notes based on victims' files. No human required.

Dubbed PromptLock, this malware acted differently every time it ran, making it nearly impossible to detect using traditional methods. The researchers declared they'd found the first example of AI-powered ransomware, sparking global headlines.

But there was a twist. The next day, a New York University team revealed the truth: PromptLock wasn't a real attack—it was their research project, designed to prove that fully automated ransomware campaigns were possible.

The Real Criminals Are Already Here

While PromptLock was academic, actual cybercriminals have been embracing AI tools since ChatGPT's debut. The numbers tell the story.

Microsoft blocked $4 billion worth of scams and fraudulent transactions in the past year, with "many likely aided by AI content." Researchers from Columbia University and the University of Chicago found that at least 50% of spam emails now use large language models. Targeted attacks jumped from 7.6% in April 2024 to 14% by April 2025.

The most dramatic case? An Arup engineering firm employee transferred $25 million to criminals during a video call with deepfake versions of colleagues, including the CFO. Every person on the call was AI-generated.

From Debugging to Destruction

Google's Threat Analysis Group tracked the evolution. Initially, hackers used Gemini like any other user—debugging code, automating tasks, writing the occasional phishing email. By 2025, they'd progressed to creating new malware with AI assistance.

One China-linked group tricked Gemini into revealing system vulnerabilities by pretending to participate in a cybersecurity competition. The AI initially refused on safety grounds but eventually complied with the deception.

The bigger concern? Open-source models. "Those are the ones bad actors will adopt because they can jailbreak them and tailor them to what they need," says Ashley Jess, senior intelligence analyst at Intel 471. The NYU team confirmed this, finding they didn't even need jailbreaking techniques with open-source models.

The Automation Arms Race

In November, Anthropic reported disrupting what it called the first "large-scale cyberattack" executed without "substantial human intervention." A Chinese state-sponsored group used Claude to automate 90% of a sophisticated espionage campaign.

But there were caveats. Humans still selected targets. Of 30 attempts, only a "handful" succeeded. Claude frequently hallucinated, fabricating credentials it hadn't obtained and overstating findings.

"None of the malicious-attack part was actually done by the AI," says veteran security expert Gary McGraw. "That stuff's been automated for 20 years."

Yet Anthropic warns this represents a fundamental shift. "We're entering an era where the barrier to sophisticated cyber operations has fundamentally lowered," says Jacob Klein, head of threat intelligence.

Defense Still Works (For Now)

The good news? Traditional defenses remain effective. Spam filters catch AI-generated phishing emails. Antivirus software detects new malware variants. The security practices we've relied on for over a decade still apply.

Ironically, AI is also strengthening defenses. Microsoft processes over 100 trillion signals daily through AI systems to detect potentially malicious activity.

But the landscape is shifting. "We're talking about someone using a scattergun approach with a model that can reasonably competently encrypt your hard drive," says Liz James from NCC Group. "You've achieved your objective."

The Billion-Dollar Question

The most extreme possibility? An AI capable of creating and automating zero-day exploits—attacks using previously unknown software vulnerabilities. But building such a system would require billions in investment, limiting it to wealthy nation-states.

Northeastern University's Engin Kirda believes it's already happening: "I'm sure people are investing in it, especially in China—they have good AI capabilities."