TLDR: A joint study by Google DeepMind and University College London found that Large Language Models (LLMs) frequently change their answers when challenged, even with weak opposing input. The research highlights that LLMs exhibit a ‘choice-supportive bias’ when their initial answer is visible and an oversensitivity to contradictory advice, unlike human confirmation bias. This behavior raises concerns for the reliability of AI systems in multi-turn interactions.
A recent collaborative study by Google DeepMind and University College London has shed light on a critical behavioral pattern in large language models (LLMs): their tendency to alter responses when challenged, even in the face of weak or incorrect opposing information. The research, published on July 16, 2025, reveals that while LLMs may initially exhibit strong confidence, this conviction can rapidly diminish when confronted with contradictory input, leading to inconsistent behavior.
The study employed a controlled experiment where models such as Gemma 3, GPT-4o, and o1-preview were tasked with answering multiple-choice questions. Following an initial response, each model received ‘advice’ from a simulated AI agent, complete with an accuracy score (e.g., 70% or 90%). This advice either supported, contradicted, or offered no input on the model’s original answer.
A key finding was the significant impact of answer visibility on the models’ willingness to change their minds. When the LLM’s initial answer was hidden during the final decision-making process, the mean change of mind rate was 32.5%. In contrast, when the initial answer was visible, this rate dropped to 13.1%. This disparity points to a ‘choice-supportive bias,’ where seeing their own answer made the models more resistant to revision.
Furthermore, the study uncovered an asymmetry in how LLMs process advice. While they did integrate external input, contradictory advice had a disproportionately stronger influence than supportive input. The researchers noted that ‘opposing advice was significantly overweighted compared to an ideal observer,’ with a measured update ratio of 2.58 in one condition. Conversely, supportive input resulted in ‘almost no unusual change’ and ‘no overweighting was observed’ in the hidden-answer condition. This suggests that LLMs overreact to disagreement but are largely unaffected by agreement.
Interestingly, the study drew parallels and distinctions between LLM behavior and human cognitive patterns. The combination of sticking to initial answers and becoming overly uncertain when challenged mirrors some human traits. However, a significant deviation was observed: unlike humans, who often exhibit confirmation bias by favoring information that aligns with their beliefs, LLMs demonstrated the opposite, overweighting opposing rather than supportive advice. This difference may stem from their training methodologies.
Also Read:
- Bridging the Trust Gap: Why AI Needs to Learn Human-Like Uncertainty
- Unpacking Emotion Hierarchies in Large Language Models
The implications for AI developers and product designers are substantial. The researchers suggest that refining how LLMs handle memory and prior responses could mitigate these biases. For instance, summarizing past exchanges and removing specific attributions might reduce the observed inconsistencies. The study emphasizes that ‘the visibility of the LLM’s own answer inflates confidence and induces a marked reluctance to change mind.’ Understanding the dynamics of confidence formation and shifting within LLMs is crucial for developing AI systems that perform predictably and reliably, particularly in complex or high-stakes applications. This research serves as a vital step towards building more robust and trustworthy conversational AI agents.
