TLDR: A new research paper introduces a framework to measure and mitigate identity bias (sycophancy and self-bias) in multi-agent LLM debates. It proposes “Response Anonymization” to remove identity markers from prompts, forcing agents to evaluate responses based on content rather than source. Experiments show that identity bias is widespread, sycophancy is dominant, and anonymization effectively reduces this bias across models and tasks without significantly impacting performance, leading to more reliable AI reasoning.
Large Language Models (LLMs) are increasingly being used in multi-agent debate (MAD) systems, where multiple AI agents exchange ideas and refine their answers to complex problems. This approach aims to leverage collective intelligence, similar to human courtrooms or scientific peer reviews, to improve reasoning and decision-making. However, recent research has uncovered a significant flaw: these AI agents are not neutral. They exhibit identity-driven biases, which can undermine the very purpose of collaborative reasoning.
A new study, titled “Measuring and Mitigating Identity Bias in Multi-Agent Debate via Anonymization,” delves into these biases, categorizing them into two main forms: sycophancy and self-bias. Sycophancy occurs when an agent uncritically adopts a peer’s view, even if its own internal beliefs are stronger. Conversely, self-bias is the tendency for an agent to stubbornly stick to its own prior outputs, disregarding valid counter-evidence from peers. While these biases have been observed in single-agent interactions, their impact on multi-agent debates has been largely unexplored until now.
The researchers, Hyeong Kyu Choi, Xiaojin Zhu, and Yixuan Li from the Department of Computer Sciences at the University of Wisconsin-Madison, introduce a comprehensive framework to understand and address this issue. First, they formalize the debate process as an identity-weighted Bayesian update, which helps model how agents’ beliefs evolve based on both content and the source of information (self or peer).
Introducing Response Anonymization
To combat identity bias, the paper proposes a simple yet powerful intervention: Response Anonymization. In typical MAD setups, agents are explicitly told whether a response came from “self” or a “peer.” This labeling creates the channel through which sycophancy and self-bias emerge. Anonymization removes these identity markers, presenting arguments without attribution. By doing so, agents are forced to weigh all responses equally, based solely on their content rather than their source. This method is remarkably minimalist, requiring no model retraining or architectural changes, making it widely applicable.
To quantify the extent of identity bias, the study defines the Identity Bias Coefficient (IBC). This metric measures how much an agent’s tendency to follow a peer versus itself is influenced by identity labels, separating it from genuine belief differences. A positive IBC indicates sycophancy, while a negative IBC points to self-bias.
Also Read:
- Self-Aware LLMs: Improving Debate Efficiency and Accuracy
- Building Trustworthy AI: How Traceability and Accountability Improve Multi-Agent LLM Systems
Key Findings from Experiments
The researchers conducted extensive experiments across various LLMs (Qwen2.5-7b-instruct, Qwen2.5-32b-instruct, Llama3.1-8b-instruct, Mistral-7b-v0.3, and GPT-OSS-20b) and benchmark datasets (GPQA, MMLU Professional Medicine, HellaSwag, and GSM8K). Their findings were striking:
- Widespread Bias: Identity bias is prevalent across different models and tasks.
- Sycophancy Dominates: In most cases, sycophancy (overweighting peer responses) was far more common than self-bias. Out of 20 evaluated scenarios, 18 showed positive IBC values.
- Anonymization Works: Response anonymization consistently and significantly reduced identity bias. For instance, on MMLU, Qwen-32B’s bias measure dropped from 0.608 to 0.024 after anonymization, a near-complete removal of identity-driven distortion.
- Performance Maintained: Crucially, removing identity bias through anonymization did not severely distort task performance, often keeping it similar to the biased setting. This suggests that the intervention improves the reliability of reasoning without sacrificing accuracy.
- Bias Amplifies: The study also found that identity bias tends to increase in subsequent debate rounds, indicating a compounding effect that anonymization can prevent.
- Heterogeneous Agents: Even when agents had distinct personas (e.g., Doctor, Programmer), identity bias persisted, though it was slightly reduced compared to homogeneous agents. Anonymization remained effective in these diverse settings.
This research highlights a critical need to ensure that multi-agent debate systems reason based on the substance of arguments rather than the identity of their source. By masking identity, AI debates can become more reliable and aligned with their intended purpose of error correction and diverse reasoning. For more details, you can read the full paper here.
