TLDR: A new research paper introduces a reliable pipeline to detect entire websites dominated by LLM-generated content. Unlike previous methods that struggle with diverse web text, this pipeline filters for prose-like content, aggregates scores from multiple pages, and uses machine learning to classify sites with high accuracy. Findings show a significant and growing presence of LLM-dominant sites in search results and web archives, raising concerns about content quality and search engine integrity.
In an era where artificial intelligence is increasingly shaping our digital landscape, a new challenge has emerged: distinguishing human-created web content from that generated by large language models (LLMs). As LLMs become more sophisticated, they are being used to automatically produce vast amounts of web content with minimal human oversight. This “LLM-dominant” content, while efficient to produce, often comes with significant drawbacks, including potential inaccuracies, plagiarism, and even hallucinations, making it unreliable and raising ethical concerns.
The problem is compounded by the fact that websites rarely disclose when their content is AI-generated, and human readers find it incredibly difficult to tell the difference. This pressing issue highlights the need for robust and reliable methods to detect LLM-dominant content, especially at the scale of entire websites.
Current LLM detectors, while advanced, primarily perform well on clean, well-structured text. They struggle with the diverse formats and complex markups found across the web, such as link listings, privacy notices, or short, non-prose texts. This limitation means that a simple page-by-page analysis often yields inaccurate results when applied to the real-world web.
A Novel Approach to Website Detection
Researchers from the University of Southern California – Sichang “Steven” He, Ramesh Govindan, and Harsha V. Madhyastha – have proposed a highly reliable and scalable pipeline designed to classify entire websites. Their innovative approach moves beyond simply analyzing individual text snippets. Instead, it classifies a website based on the aggregated outputs of an LLM text detector applied to multiple prose-like pages from that site. You can read their full research paper for more details: Did I Just Browse A Website Written by LLMs?
The pipeline works by first sampling pages from a website, either from its sitemap or web archives. It then extracts the main textual content from these pages. A crucial step involves rigorous filtering: pages with short texts, high ratios of text in lists or tables, or duplicated content are discarded. This ensures that the remaining content is primarily prose-like, allowing existing LLM text detectors, like Binoculars (which the researchers found to be highly effective), to perform optimally.
To overcome the inaccuracies of single-page detection and the inherent “noise” of web content, the pipeline aggregates scores from at least 15 filtered pages per site. By analyzing the distribution of these scores, a more robust signal is obtained. This aggregated data is then fed into a linear support vector machine (SVM), which classifies whether the entire website is LLM-dominant.
Groundbreaking Evaluation and Real-World Findings
To train and test their detector, the researchers created two distinct ground truth datasets totaling 120 websites. One dataset, “Company,” included human-written company sites and corresponding LLM-generated sites from Wix.com. The “Personal” dataset comprised human personal blogs and LLM-generated counterparts from B12.io. Remarkably, their detection pipeline achieved 100% accuracy when tested across these datasets, even when trained on one and tested on the other, demonstrating its robustness against out-of-distribution data.
Applying their trained detector to real-world scenarios yielded significant insights. When analyzing over 10,000 sites from Bing search results, a substantial 9.96% were detected as LLM-dominant. These sites often appeared as generic blogs filled with advertisements and attributed to fabricated authors. Interestingly, the study found no statistically significant difference in search rankings between LLM-dominant and human-dominant sites, suggesting that search engines like Bing may not be effectively penalizing AI-generated content, potentially impacting user experience and the quality of search results.
Furthermore, an analysis of over 10,000 random sites from Common Crawl archives (dating from 2020 to 2025) revealed a clear trend. While the overall percentage of LLM-dominant sites was lower than in search results, a significant increase was observed in sites crawled after ChatGPT’s launch. For instance, 7.25% of sites crawled entirely after ChatGPT launched were LLM-dominant, rising to 10.08% for sites crawled between 2024 and 2025. This indicates a growing prevalence of AI-generated content on the web.
Also Read:
- VLA-Mark: Securing AI-Generated Multimodal Content with Vision-Aligned Watermarks
- Building Reliable AI: Bridging Large Language Models and Expert Systems
Implications for the Web Ecosystem
The findings from this research raise important questions about the future of web content. As LLM-dominant sites become more common and rank highly in search results, there are concerns about their impact on end-users and the overall integrity of the web. While some of these sites appear to be designed for ad revenue, understanding the motivations and methods behind their creation is crucial.
The researchers plan to continue investigating challenges such as false positives and “in-between” sites (those partially composed of LLM-dominant pages). Their work paves the way for better evaluation methods and a deeper understanding of the broader impact of AI-generated content on the quality of search engine results and the digital ecosystem as a whole.
