TLDR: A new research paper demonstrates that large language models (LLMs) can effectively simulate human behavior in various auction formats, replicating established economic findings such as risk-averse bidding, adherence to strategy-proof mechanisms, and susceptibility to the winner’s curse. The study highlights the cost-effectiveness of using LLMs for large-scale auction experiments and shows how specific prompting techniques can significantly influence their strategic decision-making, offering a new paradigm for mechanism design research.
A new research paper explores a fascinating intersection of artificial intelligence and economics, investigating how large language models (LLMs) can behave as participants in auctions. This study introduces a novel way to generate synthetic data, making it significantly easier and cheaper to study and design auction mechanisms.
The researchers found that LLMs, especially when equipped with ‘chain of thought’ reasoning, exhibit behaviors in classic auction formats that align with findings from human experimental literature. For instance, LLM bidders tend to act like risk-averse human bidders. They also perform closer to theoretical predictions in auctions that are ‘obviously strategy-proof,’ meaning their optimal strategy is clear and simple. Interestingly, these AI agents also fall victim to the ‘winner’s curse’ in common value settings, a well-known non-rational phenomenon where the winning bidder overestimates the value of the item.
One of the most significant findings is the cost-effectiveness of using LLMs for auction experiments. The study ran over 1,000 auctions with GPT-4 models for less than $400, which is orders of magnitude cheaper than traditional human-subject experiments. This cost reduction opens up new possibilities for extensive experimental study in auction design.
The paper also delves into how different prompts influence LLM behavior. Simple changes in language or currency in prompts did not significantly alter their bidding strategies. However, providing LLMs with a specific ‘mental model,’ such as the language of Nash deviations (thinking about what happens if they bid up or down), dramatically improved their performance towards theoretical predictions. This suggests that how we frame economic problems for AI agents can profoundly impact their strategic play.
Beyond classic auction formats, the research simulated an environment inspired by eBay’s online marketplace. They observed that LLM agents naturally replicate real-world bidding behaviors like ‘bid sniping’ (delaying bids until the last second) when auctions have a hard closing time. When a ‘soft-close’ rule (extending the auction if new bids come in at the last minute) was introduced, bid sniping significantly reduced, and price discovery improved, echoing dynamics seen in early online auctions between platforms like Amazon and eBay.
The ability to interact directly with these simulated agents also allowed the researchers to test interventions designed to improve the LLMs’ understanding of auction rules and economic logic. Interventions that clarified the underlying economics, particularly the ‘Nash-deviation’ prompt, were most effective in guiding LLMs toward optimal play. This highlights the potential for LLMs to serve as a testbed for designing clearer and more effective mechanisms for human participants.
Also Read:
- Navigating Strategic AI Interactions: Introducing the LLM-Nash Game Framework
- Modeling Strategic Interactions with AI: Introducing LLM-Stackelberg Games
This work provides a robust framework for using LLM experimental agents as proxies for human agents, offering a low-cost, scalable method for generating synthetic data in economic research. For more details, you can read the full research paper here.
