TLDR: Generative AI is dramatically escalating the scale and sophistication of online fraud, leading to a surge in scams like deepfake phishing, AI-powered ransomware, and pharmaceutical fraud. Global scam losses hit $1 trillion in 2024, with a 50% spike in blocked scams in Q1 2025. Despite this, consumer concern is waning, creating a ‘digitalization paradox’. However, the same AI technologies are also being leveraged by cybersecurity firms to develop advanced, real-time detection and prevention strategies to combat these evolving threats.
The rapid advancement of generative artificial intelligence (GenAI) has ushered in a new era of digital challenges, creating a ‘digitalization paradox’ where the very technology driving innovation is also fueling an unprecedented surge in sophisticated fraud. A recent Q2 2025 Threat Report by Gen Digital and analysis from Sift highlight how cybercriminals are leveraging AI to personalize, scale, and evade detection, making online fraud more believable and dangerous than ever before.
Global scam losses reached an alarming $1 trillion in 2024, with Sift’s Global Data Network reporting a 50% spike in blocked scams in Q1 2025 compared to the previous year. This acceleration is attributed to scammers utilizing GenAI to clone voices, create deepfakes, build fake AI platforms, and launch hyper-personalized phishing campaigns that can deceive even tech-savvy users.
Gen Digital’s Q2 2025 Threat Report details several critical areas of AI-fueled fraud:
1. Pharmaceutical Fraud: Over 5,000 fake pharmacy domains were uncovered, promoting high-demand medications. These operations employ polished websites, AI-generated review content, malicious code injection, and search manipulation to appear legitimate. Gen blocked a million attacks tied to these domains in Q2 2025, exposing a global scheme designed to harvest financial and personal data.
2. AI-Powered Ransomware: The report highlighted the takedown of ‘FunkSec,’ the first ransomware strain partially developed with generative AI. While capable of encrypting data, Gen researchers found a cryptographic weakness, allowing victims to recover files without paying ransom. This case demonstrates that even AI-enhanced malware can contain exploitable flaws.
3. Financial and Sextortion Scams: From April to June 2025, financial scams surged by 340%, often linked to fake ads and pages on platforms like Facebook. These scams utilize deepfake videos and chatbot forms to trick individuals into revealing sensitive information for fraudulent investment or legal help. Sextortion scams also doubled, increasingly targeting mobile users with urgent, manipulative scripts.
4. Technical Support Scams: These scams saw a nearly 65% global increase, with 14% of blocked threats originating from Facebook. Fake Messenger-style pop-ups would lock browsers, demanding users call fraudulent support lines.
The broader threat landscape includes a 21% increase in data breach events, a 317% spike in malicious push notifications, and a 62% rise in remote access attacks. These figures collectively paint a picture of an increasingly automated, AI-powered, and socially engineered cybercrime ecosystem.
Compounding the problem is a worrying trend in consumer behavior. Sift’s research indicates that 70% of consumers find scams harder to spot, yet a third remain confident in their ability to detect AI-generated fraud. This disconnect is particularly prevalent among younger generations (Gen Z and Millennials), whose familiarity with GenAI may be fostering a false sense of security. Alarmingly, 31% of consumers admit to entering personal or sensitive data into GenAI platforms.
However, the narrative is not entirely bleak. The same AI technologies empowering criminals are also being harnessed by defenders. As Allen, a fraud-fighting expert quoted by Sift, noted, “There’s very few times in that history when there have been big shifts in the tools that are available and technology changing. This is one of them.”
Cybersecurity firms like Gen Digital (with brands like Norton and Avast) and Sift are deploying AI-powered solutions for real-time scam detection, device protection, and digital identity trust. These adaptive solutions are designed to understand user behavior in context, detect fraud before it happens, and automate decisions. Sift’s ‘Activity IQ,’ for instance, leverages GenAI to investigate account takeovers, reducing investigation time by up to 90%.
Also Read:
- SpamGPT: AI-Powered Tool Revolutionizes Cybercrime, Lowering Barriers for Phishing and Ransomware
- Combating Digital Deception: The Rise of AI Deepfake Detection for Social Media Integrity
To stay protected, experts advise questioning ‘too good to be true’ offers, trusting comprehensive security tools, limiting data exposure on social platforms, and exercising caution with unsolicited tech support. The ongoing ‘AI arms race’ in cybersecurity underscores the critical need for continuous research, collaboration, and proactive defense strategies to safeguard the digital environment against evolving AI-driven threats.
