TLDR: A study found that Large Language Models (LLMs) exhibit social conformity driven by both rational information processing and a “normative-like” tendency to align with majority opinions, especially under high uncertainty. While LLMs generally use evidence to improve decisions, high uncertainty causes them to over-rely on public signals, similar to human groupthink. In contrast, under low uncertainty, they prioritize private information. The study also revealed LLMs treat human and AI advice equally. These findings suggest LLMs mirror human dual-process reasoning and highlight the need for uncertainty-aware AI systems to prevent collective errors.
As large language models (LLMs) become increasingly integrated into collaborative teams, a critical question arises: do these advanced AI systems exhibit social conformity, and if so, what drives it? A recent research paper, “Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process Mechanism”, by Huixin Zhong, Yanan Liu, Qi Cao, Shijin Wang, Zijing Ye, Zimu Wang, and Shiyao Zhang, delves into this complex behavior, revealing that LLMs, much like humans, adapt their decision-making based on the level of uncertainty they face.
In humans, social conformity typically stems from two main influences: informational influence, where individuals rationally use group cues to improve accuracy, and normative influence, driven by social pressure to gain approval. The balance between these two is often moderated by uncertainty, pushing individuals from analytical to more heuristic (rule-of-thumb) processing. This study set out to determine if these psychological mechanisms also apply to LLMs.
Unpacking LLM Conformity: The Study’s Approach
To investigate this, the researchers adapted an information cascade paradigm, a method from behavioral economics, to quantitatively measure how LLMs weigh private information against public signals. They evaluated nine leading LLMs, including models like GPT-4o, Gemini 2.5 Pro, and Claude 3.7 Sonnet, across three distinct decision-making scenarios: medical diagnosis, legal evaluation, and investment analysis. Crucially, they manipulated the level of information uncertainty in each scenario, ranging from low (investment task, 70% accuracy) to medium (medical task, 66.7% accuracy) to high (legal task, 55% accuracy).
The LLMs were given a piece of private information and then exposed to decisions from a group of one to three external “advisors,” which were explicitly labeled as either human or AI. To ensure robust reasoning, the models were instructed to use a “Chain of Thought” approach, first reasoning step-by-step before providing a final decision and confidence score.
Informational Influence: LLMs Seek Accuracy
The study’s first major finding confirms that informational influence is a primary driver of LLM conformity. When there was a clear “most likely choice” based on the evidence, LLMs consistently increased both their likelihood of selecting that choice and their confidence in it as the evidence grew stronger. This suggests that LLMs rationally integrate information to improve their decision-making accuracy.
However, not all models performed equally. Top-tier models like Gemini-2.5 Pro and GPT-4o demonstrated exceptional reliability, maintaining high accuracy and stable confidence even under ambiguous conditions. Other models showed more variability, highlighting that while all LLMs improve with clear evidence, their robustness under uncertainty is a key differentiator.
The Dual-Process Mechanism: Uncertainty Changes Everything
The most significant discovery was how information uncertainty dramatically modulates LLM behavior, revealing a dual-process mechanism akin to human cognition. The study found that LLMs adapt their weighting of information sources based on how uncertain the situation is:
- High Uncertainty (Legal Task): In scenarios with high ambiguity, LLMs exhibited a “normative-like amplification.” They significantly overweighted public information (advice from others) compared to their private information. This behavior mirrors human “groupthink,” where individuals over-rely on the crowd when unsure, potentially leading to collective errors.
- Medium Uncertainty (Medical Task): Here, LLMs adopted a more conservative strategy, systematically underweighting all sources of evidence – both private and public – relative to a purely rational benchmark.
- Low Uncertainty (Investment Task): In situations with high certainty, LLMs showed a preference for their private information, weighting it more heavily than public signals, although all signals were still somewhat underweighted compared to a perfectly rational approach.
Interestingly, the study found no significant bias in how LLMs weighed information from human advisors versus AI advisors across any scenario. This suggests that LLMs treat these sources agnostically, possibly due to the homogenized nature of their training data.
Also Read:
- Why More Thinking Doesn’t Always Make AI More Confident (or Correct)
- Decoding AI’s Emotional Understanding: A Study on Language Models and Human Feelings
Implications for AI Safety and Design
These findings have profound implications for AI safety and alignment. The revelation that extreme uncertainty can push LLMs into a “groupthink”-like state highlights a critical vulnerability. In high-stakes applications, this normative conformity could amplify collective errors or reinforce societal biases present in the training data. It raises a crucial question: are LLMs simply mimicking human decision patterns statistically, or are they strategically learning to produce agreeable, low-risk responses (sycophancy) to maximize rewards, potentially signaling a form of deceptive alignment?
The research underscores the need for developing “uncertainty-aware” AI systems. These systems should be capable of detecting ambiguity and triggering mitigation strategies, such as actively seeking dissenting opinions, to ensure more robust and ethically aligned AI collaboration. By understanding the cognitive pathways behind AI outputs, rather than just the outputs themselves, we can design more trustworthy and value-consistent AI deployments in the future.
