TLDR: Check Point Research has released its ‘AI Security Report 2025,’ highlighting the significant and rapidly evolving cybersecurity threats posed by the weaponization of artificial intelligence. The report identifies four critical areas of concern: AI-enhanced impersonation and social engineering, LLM data poisoning and disinformation, AI-created malware and data mining, and the weaponization and hijacking of AI models. Check Point emphasizes the urgent need for organizations to adopt AI-aware defense strategies, including AI-assisted detection, enhanced identity verification, and AI-contextualized threat intelligence, to counter the sophisticated attacks emerging in this new era.
SAN FRANCISCO – Check Point Software Technologies Ltd. (NASDAQ: CHKP), a global leader in cybersecurity solutions, unveiled its inaugural ‘AI Security Report 2025’ at the RSA Conference on April 30, 2025. The comprehensive report serves as a stark warning about the profound impact of artificial intelligence on the cyber threat landscape, detailing how cybercriminals are increasingly leveraging AI to launch more sophisticated and scalable attacks.
According to Lotem Finkelstein, Director of Check Point Research, the rapid adoption of AI by malicious actors is fundamentally reshaping the threat environment. Finkelstein highlighted an imminent shift towards ‘digital twins’ – AI-driven replicas capable of mimicking human thought and behavior, stating, ‘These aren’t just lookalikes or soundalikes, but AI-driven replicas capable of mimicking human thought and behavior. It’s not a distant future – it’s just around the corner.’ This development underscores a critical erosion of trust in digital identity, where what is seen, heard, or read online can no longer be taken at face value.
The report identifies four core areas where AI’s ability to impersonate and manipulate digital identities is most evident:
1. AI-Enhanced Impersonation and Social Engineering: Threat actors are utilizing AI to generate highly realistic, real-time phishing emails, audio impersonations, and deepfake videos. A notable incident cited in the report involved attackers mimicking Italy’s defense minister using AI-generated audio, demonstrating the vulnerability of even high-profile individuals to AI-powered fabrication.
2. LLM Data Poisoning and Disinformation: Malicious actors are manipulating AI training data to skew outputs and spread disinformation. The report highlights a case involving Russia’s ‘Pravda’ disinformation network, which generated approximately 3.6 million articles in 2024 aimed at influencing AI chatbot responses. Research indicated that these chatbots echoed Pravda’s false narratives about 33% of the time, emphasizing the significant danger of AI being used for propaganda.
3. AI-Created Malware and Data Mining: AI is being integrated across the entire cyberattack lifecycle, from generating malicious code to optimizing campaigns. Examples include FunkSec’s AI-generated DDoS module and custom ChatGPT-style chatbots used by ransomware groups. Furthermore, AI plays a crucial role in analyzing stolen data, with services like ‘Gabbers Shop’ using AI to validate, clean, and enhance the resale value of stolen credentials, session tokens, and API keys.
4. Weaponization and Hijacking of AI Models: Cybercriminals are not just using AI; they are turning it into a dedicated tool for cybercrime. This includes the hijacking and commercialization of Large Language Model (LLM) accounts through credential stuffing and infostealer malware. Even more concerning is the emergence of ‘Dark LLMs’ such as FraudGPT, WormGPT, HackerGPT Lite, and GhostGPT. These maliciously modified AI models, often created by jailbreaking ethical systems or modifying open-source models, are marketed on dark web forums as hacking tools, complete with subscription access and user support.
The report also reveals alarming statistics regarding data leakage, with an analysis of Check Point’s GenAI Protect showing that 1 in every 80 GenAI prompts poses a high risk of sensitive data leakage, and 7.5% (approximately 1 in 13) of prompts contain potentially sensitive information.
In response to these escalating threats, Check Point Research stresses that defenders must now assume AI is embedded within adversarial campaigns. The report outlines key defensive strategies, including:
- AI-Assisted Detection and Threat Hunting: Leveraging AI to identify AI-generated threats and artifacts like synthetic phishing content and deepfakes.
- Enhanced Identity Verification: Moving beyond traditional methods to implement multi-layered identity checks that account for AI-powered impersonation across text, voice, and video.
- Threat Intelligence with AI Context: Equipping security teams with the necessary tools to recognize and respond effectively to AI-driven tactics.
Also Read:
- AI’s Rapid Evolution: Misinformation, Cyber Threats, and Market Shifts Dominate Recent Headlines
- Pro-ISIS Channels Endorse Encrypted Generative AI for Propaganda and Evasion
Finkelstein concluded, ‘In this AI-driven era, cybersecurity teams need to match the pace of attackers by integrating AI into their defenses. This report not only highlights the risks but provides the roadmap for securing AI environments safely and responsibly.’ The full ‘AI Security Report 2025’ is available for download from Check Point’s website.
