TLDR: Amazon has significantly ramped up its efforts to prevent AI bots from major tech companies like Meta, Google, and Huawei from accessing its e-commerce platform. This move, highlighted by recent updates to its robots.txt file on August 21, 2025, is a strategic defensive measure in the burgeoning ‘AI commerce wars,’ as Amazon seeks to protect its vast data and control the development of AI-powered shopping agents.
Amazon, the global e-commerce behemoth, has intensified its campaign to restrict artificial intelligence companies from scraping data from its vast online marketplace. On August 21, 2025, the retail giant updated its robots.txt file, adding six new AI-related crawlers to its publicly available restrictions, specifically targeting bots from Meta, Google, Huawei, and Mistral, among other technology firms . This action builds upon earlier measures implemented at least a month prior, which blocked crawlers from Anthropic’s Claude, Perplexity, and Google’s Project Mariner agents .
The move underscores an escalating ‘AI commerce battle,’ as major tech players race to develop their own AI shopping agents capable of browsing, comparing, and purchasing products across various platforms. Amazon’s stance is clear: it aims to prevent rivals from training their AI models on its proprietary e-commerce data, thereby safeguarding its competitive edge and ensuring it controls the future of AI-driven shopping experiences .
Independent analyst Juozas Kaziukėnas, who first identified these changes, commented, ‘Amazon is desperately trying to stop AI companies from training models on its data.’ He further noted, ‘I think it is too late to stop AI training — Amazon’s data is already in the datasets ChatGPT and others are using. But Amazon is definitely not interested in helping anyone build the future of AI shopping. If that is indeed the future, Amazon wants to build it itself’ .
This aggressive defensive strategy by Amazon contrasts with the approaches of other major retailers. A review of robots.txt files by Modern Retail indicates that Walmart and eBay have not implemented similar restrictions against AI bots, suggesting a divergence in how companies are navigating the balance between competitive positioning and the potential benefits of AI-powered shopping tools .
The ‘AI commerce wars’ have been brewing for some time. The trend of blocking AI crawlers gained momentum shortly after OpenAI announced GPTBot on August 7, 2023, leading to major sites like Amazon, Quora, The New York Times, and CNN implementing blocks within two weeks . In July 2024, Shopify introduced robots.txt warnings for merchants regarding ‘buy-for-me’ agents, and by July 31, 2024, HUMAN Security reported that 80% of companies were blocking AI language models . McKinsey also identified ‘agentic AI’ as a top trend reshaping marketing in July 2025 .
Amazon is not merely playing defense; it is also actively developing its own AI capabilities. The company is testing its generative shopping assistant, ‘Rufus,’ and a ‘buy-for-me’ feature that allows customers to purchase items from third-party websites without leaving the Amazon app . This indicates Amazon’s desire to own the entire customer relationship – from discovery and decision-making to payment and loyalty – in the age of agentic shopping .
Also Read:
- Retailers Intensify AI Agent Deployment for Peak Holiday Shopping Season
- Walmart’s AI-Powered Transformation: Reshaping Retail Operations and Customer Experience
While current robots.txt files serve as advisory guidelines for ‘well-behaved’ crawlers, the long-term effectiveness of such blocks remains a subject of debate. Much of Amazon’s catalog data may already be incorporated into existing AI training datasets . Experts like Karsten Weide predict that blocking shopping bots will ultimately be a ‘losing defensive battle,’ suggesting that the future of AI commerce may shift from outright blocking to brokering controlled access via secure APIs and new interoperability standards .
