TLDR: Researchers have developed a new AI framework that teaches large language model agents to be more persuasive in social deduction games like Werewolf and Avalon. By modeling conversations as a strategic “leader-follower” interaction, the AI learns to craft utterances that influence other players’ responses, leading to significantly higher win rates compared to existing methods.
Large language models (LLMs) have shown impressive capabilities in various tasks, including playing complex games. However, a new research paper highlights a crucial area where these AI agents often fall short: persuasive communication in social deduction games (SDGs).
Social deduction games, such as Werewolf, Avalon, and One Night Ultimate Werewolf (ONUW), are multiplayer experiences where players have hidden roles and must use communication, deduction, and deception to achieve their goals. Success in these games isn’t just about making correct logical deductions; it’s also about convincing other players to act in alignment with one’s own intentions.
The paper, titled “LEADING THE FOLLOWER: LEARNING PERSUASIVE AGENTS IN SOCIAL DEDUCTION GAMES,” addresses this gap by introducing a novel approach. The authors, Zheng Zhang, Deheng Ye, Peilin Zhao, and Hao Wang, propose a reinforcement learning framework that trains AI agents to optimize their spoken statements for maximum persuasive impact. You can find the full research paper here: RESEARCH_PAPER_URL.
A New Way to Model Conversations
The core idea behind this research is to formalize turn-based dialogue in social deduction games as a “Stackelberg competition.” Imagine a scenario where one player (the “leader”) makes a move first, and another player (the “follower”) then responds after observing the leader’s action. In this model, the current player speaking acts as the leader, strategically influencing the next player’s response.
This framework allows the leader to anticipate how the follower might react to different statements. If the leader understands the follower’s likely responses, they can choose their words to achieve the best possible outcome for themselves.
How the Persuasive Agents Are Trained
The training process involves three main steps:
- Intent Identification: The AI agent first analyzes the current game situation to determine what response from the next player would be most beneficial (desired) and most harmful (undesired) for its win condition.
- Impact Measurement: The agent then generates a base statement using a powerful, off-the-shelf LLM. This base statement is then refined by a specialized “Refiner” model. The persuasive impact of these refined statements is measured by how much they shift the probability of the follower generating the desired response versus the undesired one.
- Strategy Optimization: Using a technique called Group Relative Policy Optimization (GRPO), the Refiner model is fine-tuned. This allows the agent to learn how to craft utterances that proactively guide conversations towards favorable outcomes, without needing explicit human-annotated preference data.
Interestingly, the system uses a two-stage approach: a high-capability, closed-source LLM (like GPT-4o or Gemini-2.5-Flash) generates the initial, logical statements, while a fine-tuned open-source LLM (like Llama-3-8B-Instruct) acts as the “Refiner” to make those statements more persuasive.
Impressive Results Across Games
The researchers conducted extensive experiments across Werewolf, Avalon, and ONUW. Their agents consistently outperformed existing baseline methods, achieving higher win rates. This improvement was observed even when the persuasive module was integrated with already strong baseline agents, suggesting it complements existing strategies by adding a crucial layer of social influence.
An ablation study, which tested different components of the reward function, showed that considering both desired and undesired responses from the follower was most effective. Simply trying to avoid undesired responses wasn’t enough; actively steering towards desired ones was key.
Furthermore, the trained Refiner demonstrated strong generalizability, improving performance even when paired with LLMs (like GPT-5 and Qwen3-14B) that were not part of its initial training data. This indicates that the framework learns fundamental principles of persuasion rather than just overfitting to specific language models.
Also Read:
- Information Theory Framework Unlocks Secrets of True Collaboration in Multi-Agent AI Systems
- Unlocking Logic: How Small Language Models Learn Deductive Reasoning
Implications for AI and Human Interaction
This work represents a significant step towards developing AI agents capable of strategic social influence. By understanding and optimizing persuasive communication, these agents can navigate complex human-like interactions more effectively. The implications extend beyond games, potentially impacting scenarios that require nuanced communication and social influence in real-world applications.
