TLDR: LLM-HANABI is a new benchmark using the cooperative card game Hanabi to evaluate how Large Language Models (LLMs) infer rationales and apply Theory-of-Mind (ToM) in dynamic, multi-agent settings. The study found a strong positive correlation between ToM proficiency and game success. Notably, the ability to accurately interpret a partner’s direct intent (first-order ToM) was a more significant predictor of performance than predicting how a partner would interpret a hint (second-order ToM), suggesting a focus on first-order ToM for enhancing collaborative AI.
In the rapidly evolving landscape of artificial intelligence, Large Language Models (LLMs) have demonstrated remarkable capabilities in logical inference and reasoning. However, their true potential in multi-agent collaboration, especially when it comes to understanding the intentions behind others’ actions – a concept known as Theory-of-Mind (ToM) – remains a significant area of exploration. A new study introduces LLM-HANABI, a novel benchmark designed to rigorously evaluate these critical aspects in LLMs within a dynamic, cooperative game setting.
The research, conducted by Fangzhou Liang, Tianshi Zheng, Chunkit Chan, Yauwai Yim, and Yangqiu Song from the Department of Computer Science and Engineering, HKUST, addresses a crucial gap in existing ToM benchmarks. Many current evaluations rely on static, text-based tasks that don’t fully capture the complexities of real-world, interactive collaboration where information is often imperfect and dynamic. LLM-HANABI leverages the cooperative card game Hanabi to create an environment where agents must infer rationales from sparse linguistic hints to succeed.
What is Hanabi and Why is it Ideal for ToM Evaluation?
Hanabi is a unique cooperative card game where players cannot see their own cards. The objective is to build five color-coded stacks of cards in ascending order (1 to 5). Players must rely on teammates to give hints about their cards’ colors or ranks. This setup inherently creates a game of imperfect information, forcing players to deduce intentions and information from limited communication. Unlike adversarial games, Hanabi’s purely cooperative nature isolates the challenge of collaborative reasoning, making it a perfect testbed for evaluating an agent’s ability to infer and act on a partner’s intentions.
How LLM-HANABI Evaluates Theory-of-Mind
The LLM-HANABI framework features an automated evaluation system that measures both game performance and ToM proficiency through a two-phase process: reasoning extraction during gameplay and post-game scoring. During each hint action, agents generate structured statements:
- Rationale: The hinting player’s justification for their hint, serving as the ground truth for their intent.
- First-Order ToM: The hinted player’s interpretation of the hinter’s intent.
- Second-Order ToM: The hinting player’s prediction of how the hinted player will interpret the hint.
After each game, an LLM-as-a-judge evaluates these statements to assign ToM scores. The First-Order ToM Score measures how accurately the recipient inferred the hint’s rationale, while the Second-Order ToM Score assesses the hinter’s accuracy in predicting the recipient’s interpretation.
Also Read:
- Beyond Mimicry: Rethinking AI’s ‘Theory of Mind’ and the Path to Mutual Understanding
- Boosting Teamwork in AI: How Prompt Engineering and LLMs Enhance Collaborative Agents
Key Findings: First-Order ToM is Crucial
The study benchmarked a diverse set of LLMs and Large Reasoning Models (LRMs), revealing significant insights:
- Superior LRM Performance: Large Reasoning Models generally outperformed standard LLMs in both gameplay and ToM assessments, indicating their enhanced capacity for effective cooperation. Models like Deepseek-R1 and QwQ-32B showed strong results, with GPT-4.1 and GPT-4.1-mini also performing exceptionally well among LLMs.
- First-Order ToM Dominance: Across all models, first-order ToM scores were consistently and significantly higher than second-order scores. This suggests that while models are good at interpreting direct intent, they struggle more with predicting how others will interpret that intent.
- Strong Correlation with Game Success: A strong positive correlation was found between ToM proficiency and game performance. Crucially, first-order ToM correlated more strongly with game success (r=0.76) than second-order ToM (r=0.58). This highlights that an agent’s ability to accurately interpret a partner’s rationale is more critical for successful collaboration than its ability to predict how its partner will interpret its own hint.
These findings suggest a clear direction for developing more effective collaborative AI: prioritizing an agent’s ability to accurately interpret its partners’ intentions (first-order ToM) is more impactful than focusing on higher-order reasoning about others’ beliefs. The LLM-HANABI benchmark provides a valuable tool for future research to develop and evaluate methodologies aimed at enhancing these specific inferential skills. You can find the full research paper here.
