TLDR: A UC San Diego study found that Large Language Models (LLMs) frequently generate content that induces cognitive biases in users. LLMs alter sentiment in 21.86% of summaries, exhibit primacy bias in 5.94% of cases, and hallucinate on post-knowledge-cutoff data in 57.33% of instances. The research also identifies effective mitigation strategies, such as “Epistemic Tagging” for hallucination and “Weighted Summaries” for positional bias, highlighting the need for safeguards in AI applications.
Large Language Models (LLMs) are becoming an integral part of our daily lives, assisting with tasks from summarizing documents to supporting medical diagnoses. While these AI models are highly capable, a recent study from UC San Diego sheds light on a critical concern: their potential to inadvertently transfer cognitive biases to users. This research investigates the extent and nature of biased content generated by LLMs and its potential impact on human decision-making.
Understanding LLM-Induced Biases
- Framing Bias: This occurs when LLMs alter the sentiment or presentation of information, leading users to make different choices than they would have with the original content. For example, a neutral review might be summarized with a negative tone, influencing a user’s perception.
- Primacy and Recency Bias: LLMs might disproportionately emphasize information found at the beginning (primacy) or end (recency) of a source text, regardless of its overall importance. This can lead to users focusing on specific parts of the content, potentially overlooking other crucial details.
- Confirmation and Authority Bias: When LLMs hallucinate or generate incorrect information, especially on topics beyond their training data cutoff, users might trust this information due to the model’s perceived authority or because it confirms their existing beliefs. This can lead to misinformed decisions.
Key Findings on LLM Biases
The study evaluated several LLM families (including Llama and GPT-3.5-turbo) across summarization and news fact-checking tasks. The results revealed significant levels of bias-inducing content:
- LLMs changed the sentiment or framing of the original content in 21.86% of cases. This means that about one in five summaries presented a different emotional tone than the source material.
- In summarization tasks, LLMs exhibited primacy bias in 5.94% of cases, meaning they overly focused on the beginning of the source text.
- Perhaps most concerning, LLMs hallucinated on questions related to post-knowledge-cutoff data in a staggering 57.33% of instances. This highlights a major challenge when models are asked about very recent events not included in their training.
For instance, in Amazon product reviews, models frequently downplayed positive sentiment, shifting reviews from positive to neutral. In news interviews, shifts from neutral to negative were common. The research also showed that summaries consistently aligned more with the beginning of the source text than the middle or end, indicating a strong positional bias.
Strategies for Mitigation
The researchers explored 18 different mitigation methods to address these biases, finding that targeted interventions can be effective, though often with trade-offs:
- For Primacy Bias: Methods like “Weighted Summaries,” which allocate a fixed token budget to different sections of the text (beginning, middle, end), significantly improved overall content coverage. However, some methods designed to reduce positional bias could inadvertently worsen framing consistency.
- For Framing Bias: “Weighted Token Decoding,” which adjusts word choice during generation by down-weighting negative sentiment tokens, proved effective in reducing framing shifts. Simple “Self-Awareness Prompts” also showed minor improvements.
- For Hallucination: “Epistemic Tagging” emerged as the most effective strategy. This approach requires models to express confidence levels (e.g., “High Confidence,” “Low Confidence”) alongside their factual assertions. This forces the model to self-evaluate its knowledge and provides users with crucial signals about the reliability of the information. Other methods like “Knowledge Boundary Awareness” (explicitly defining the model’s knowledge cutoff) showed mixed results, often degrading performance in smaller models.
Also Read:
- Unpacking LLM Confidence: Why AI Models Can Be Stubborn Yet Easily Swayed
- The ‘Lost-in-the-Later’ Effect: A New Look at How LLMs Use Context
Implications for LLM Use
Given the widespread use of LLMs in critical areas like healthcare, legal analysis, and content creation, these findings underscore the urgent need for robust technical safeguards and user-centered interventions. The study emphasizes that while LLMs are powerful tools, their inherent biases can subtly but significantly influence human judgment and decision-making. The authors hope their work will encourage the development of methods to improve LLMs and educate users on how to interact with these models more thoughtfully. You can read the full research paper for more details at arXiv:2507.03194.
