TLDR: A large-scale study on a major online retail platform found that Generative AI (GenAI) significantly boosts firm productivity, increasing sales by up to 16.3% in various consumer-facing workflows. The gains primarily come from enhancing consumer experience and reducing market frictions, leading to higher conversion rates rather than increased spending per purchase. Notably, smaller sellers and less experienced consumers benefit disproportionately, highlighting GenAI’s role in bridging market capability gaps. The research suggests a substantial annual incremental value of approximately $5 per consumer from current GenAI applications.
A new research paper delves into the tangible impact of Generative Artificial Intelligence (GenAI) on the productivity of firms, specifically within the dynamic landscape of online retail. The study, titled Generative AI and Firm Productivity: Field Experiments in Online Retail, provides robust, large-scale evidence from randomized field experiments conducted on a major cross-border online retail platform.
The authors, Lu Fang, Zhe Yuan, Kaifu Zhang, Dante Donati, and Miklos Sarvary, embarked on a six-month investigation between 2023 and 2024, integrating GenAI-based enhancements into seven different consumer-facing business workflows. This extensive study involved millions of users and products, offering a comprehensive view of GenAI’s real-world effects.
Understanding the Research Approach
To accurately measure GenAI’s impact, the researchers designed their experiments such that inputs like labor, capital, and prices remained constant across experimental groups. This crucial design choice allowed any observed increases in sales to be directly attributed to improvements in total factor productivity (TFP) – essentially, getting more output from the same inputs. The workflows targeted by GenAI ranged from pre-sale service chatbots and search query refinement to product description generation, marketing push message creation, Google advertising title optimization, chargeback defense, and live chat translation. Each of these applications was evaluated through randomized field experiments, with user groups varying from tens of thousands to tens of millions.
Key Findings: Boosting Sales and Efficiency
The study revealed that most GenAI deployments led to economically significant gains, though the magnitude varied. Sales increases ranged from no detectable impact to a substantial 16.3%, depending on how much GenAI added beyond existing practices. The most significant improvements were observed in customer service and search applications. When aggregating the positive effects across four GenAI applications, the implied annual incremental value was approximately $5 per consumer. This is a meaningful impact, especially considering the retailer’s vast scale and the early stage of GenAI adoption.
Beyond sales, the research also highlighted other crucial benefits. For instance, the Chargeback Defense workflow saw a 15% increase in defense success rates, while Live Chat Translation led to a 5.2% boost in consumer satisfaction. However, not all applications yielded positive results; the Google Advertising Title workflow, for example, showed an insignificant negative effect. This was attributed to the GenAI model not being specifically fine-tuned for the advertising domain, underscoring the importance of specialized training for optimal performance.
How GenAI Drives Value: Reducing Market Frictions
A central insight from the paper is that GenAI’s productivity gains primarily stem from enhancing the consumer experience and reducing market frictions, rather than just cutting costs. The researchers found significant increases in conversion rates (the likelihood of a consumer making a purchase), ranging from 1% to 22% across different workflows. Importantly, there was no significant effect on average cart values, meaning GenAI primarily expanded the market by converting more consumers, rather than encouraging existing buyers to spend more per purchase.
GenAI achieved this by: enriching pre-sale communication through chatbots, refining search queries for better product matching, generating more comprehensive and structured product descriptions, and personalizing marketing messages at scale. These improvements collectively made the shopping experience smoother and more informative, directly contributing to higher conversion rates and sales.
Who Benefits Most? The Role of Heterogeneity
The study also uncovered significant heterogeneity in GenAI’s benefits. Smaller and less experienced sellers, as well as less experienced consumers, showed disproportionately larger gains. This suggests that GenAI plays a crucial role in bridging capability gaps across different segments of the marketplace. For instance, small sellers, often constrained by resources, benefited more from GenAI-enhanced search and marketing. Similarly, inexperienced consumers, who might struggle with information search or platform navigation, found greater value in GenAI-powered customer support and detailed product descriptions.
Product-level effects were more context-dependent, but generally, tail products (those with lower sales volume) and high-priced items often saw greater benefits, as GenAI helped overcome visibility issues or justified larger expenditures through better information.
Also Read:
- How AI Reshapes Global Competitiveness: Insights from China and the US
- GenAI Monetization: Why a Single Model and Token Caps Win
Looking Ahead
While the experiments captured short-term effects, the rapid scaling of GenAI adoption on the platform – from a handful of workflows in 2023 to over 60 by 2025, with a twentyfold increase in API calls to large language models – indicates the firm’s strong anticipation of substantial long-term value. This research provides crucial causal evidence on how GenAI can reengineer core retail workflows to deliver meaningful business outcomes, paving the way for future studies on its broader economic impacts, cost-side adjustments, and long-term productivity growth in an evolving competitive landscape.
