TLDR: AI crawlers and bots are increasingly dominating global web traffic, with some reports indicating they now account for over half of all internet activity. This surge, driven by the demands of large language models and real-time information retrieval, is causing significant disruptions, including bandwidth consumption, server strain, and new security challenges for digital platforms. Companies like Fastly and Cloudflare are highlighting the growing complexity and potential risks posed by this automated traffic.
The internet is experiencing a profound transformation as artificial intelligence (AI) crawlers and bots increasingly dominate global web traffic, leading to significant disruptions and new challenges for digital infrastructure. Recent reports from leading cybersecurity and content delivery network providers indicate that automated bot traffic has surpassed human-generated activity, fundamentally reshaping how information flows across digital networks.
According to Fastly’s Q2 2025 Threat Insights Report, released on August 20, 2025, AI crawlers constituted nearly 80% of all observed AI bot traffic between mid-April and mid-July. Meta emerged as a dominant force, generating over half of this traffic and eclipsing the combined activity of Google and OpenAI. The report also highlights the massive real-time request volumes driven by ‘fetcher bots,’ such as those used by ChatGPT and Perplexity, with some instances exceeding 39,000 requests per minute.
This unprecedented surge in AI-driven automation is placing immense pressure on unprotected origin infrastructure. It consumes vast amounts of bandwidth, overwhelms servers, and can mimic the effects of Distributed Denial of Service (DDoS) attacks, even when the bots are not intentionally malicious. The implications extend to security, performance, and operational resilience for digital platforms.
Further analysis from Cloudflare, as reported on July 2, 2025, reveals that automated bots now account for approximately 30% of all worldwide web traffic. This proliferation is directly linked to the explosive growth in large language model (LLM) development, which requires immense quantities of web data for training and refinement. Unlike traditional search engine crawlers, these new AI bots serve diverse purposes, including content analysis, model training, and real-time information retrieval. OpenAI’s GPTBot, for instance, saw explosive growth, commanding 30% of AI crawler traffic between May 2024 and May 2025, representing a 305% increase in raw request volume.
The 2025 Imperva Bad Bot Report, released by Thales CPL on April 15, 2025, corroborates this trend, stating that automated bot traffic surpassed human-generated traffic for the first time in a decade, reaching 51% of all web traffic in 2024. This report emphasizes that the rise of AI and LLMs has simplified the creation and scaling of bots, including those used for malicious purposes. Malicious bots now account for 37% of all internet traffic, a significant increase from 32% in 2023, marking the sixth consecutive year of growth in such activity.
Geographic distribution of this traffic also raises concerns. North America, for example, receives nearly 90% of observed AI crawler traffic, signaling a growing geographic bias in LLM training datasets and prompting questions about the long-term neutrality of AI model outputs.
Arun Kumar, Senior Security Researcher at Fastly, commented on the situation: ‘AI Bots are reshaping how the internet is accessed and experienced, introducing new complexities for digital platforms. Whether scraping for training data or delivering real-time responses, these bots create new challenges for visibility, control, and cost. You can’t secure what you can’t see, and without clear verification standards, AI-driven automation risks are becoming a blind spot for digital teams. Businesses need the tools and insights to manage automated traffic with the same precision and urgency as any other infrastructure or security risk.’
The increasing accessibility of AI tools has also lowered the barrier to entry for cyber attackers, enabling them to deploy a higher volume of bot attacks with increased frequency. Attackers are leveraging AI to refine their techniques and evade security measures, contributing to a growing ‘Bots-As-A-Service’ (BaaS) ecosystem.
Also Read:
- Generative AI Fuels Surge in Sophisticated Phishing Campaigns
- Generative AI Drives E-commerce Traffic Surge During Prime Day, Adobe Reports
This evolving landscape necessitates advanced strategies for digital platforms to manage automated traffic effectively, ensuring security, maintaining performance, and controlling operational costs in an increasingly AI-driven internet.
