TLDR: A research paper reveals that language models’ beliefs and behaviors are highly malleable, changing significantly as they accumulate context through interactions and reading. GPT-5 showed a 54.7% belief shift in discussions, while Grok 4 had a 27.2% shift after reading opposing political texts. These shifts are influenced by both intentional persuasion and non-intentional information exposure, raising concerns about the long-term reliability and consistency of AI assistants.
Language model (LM) assistants are becoming increasingly common in our daily lives, helping with tasks like brainstorming and research. As these models become more advanced, their ability to remember and use information from past interactions, known as “context,” has grown significantly. This means LMs can accumulate a lot of text without direct user intervention, which, as a new study reveals, carries a hidden risk: their fundamental understanding of the world, or “belief profiles,” can quietly change over time.
A recent paper titled “Accumulating Context Changes the Beliefs of Language Models” explores how these belief shifts occur when LMs engage in interactions and process text, essentially “talking and reading.” The findings highlight that the beliefs of these models are surprisingly flexible. For instance, GPT-5 showed a significant 54.7% shift in its stated beliefs after just 10 rounds of discussion on moral dilemmas and safety questions. Similarly, Grok 4 experienced a 27.2% shift on political topics after being exposed to texts from opposing viewpoints.
The researchers also investigated how these changes manifest in the models’ actions. They designed tasks requiring tool use, where each tool selection implied a certain belief. The study found that these behavioral changes aligned with the shifts in stated beliefs, suggesting that as LMs become more autonomous, these belief changes will directly influence their actions in real-world applications. This raises concerns about the reliability of LMs in extended use, as their opinions and behaviors could become unpredictable.
Understanding Belief Shift
The study defines belief shift as a change in a model’s stated preference or its choice of action after accumulating context. This context can be categorized into two types: intentional and non-intentional.
- Intentional Interactions: These involve scenarios where another agent actively tries to convince the LM to change its stance. This includes “Debate,” where two LMs argue opposing positions, and “Persuasion,” where one LM uses specific techniques (like providing information, appealing to values, norms, empathy, or elite cues) to sway another.
- Non-Intentional Exploration: These are activities where the model gathers information without explicit persuasive intent. This includes “In-depth reading” of curated documents and “Research,” where the model actively searches and studies materials from the web.
Also Read:
- Balancing Act: How Efficient Fine-Tuning Shapes LLM Safety and Fairness
- Unveiling the ‘Automatic Minds’: How Hypnosis Illuminates the Inner Workings of AI
Key Findings and Implications
The research employed a three-stage framework: first, recording the LM’s initial beliefs; second, having the LM complete various tasks to accumulate context; and third, re-evaluating its beliefs. The results consistently showed that LMs’ beliefs and behaviors are highly malleable.
In intentional tasks, GPT-5 exhibited larger shifts, especially when persuasion techniques were used, indicating its sensitivity to structured persuasive interactions. Claude-4-Sonnet, on the other hand, was more prone to belief shifts in non-intentional settings like in-depth reading, suggesting it’s more vulnerable to prolonged exposure to information. Open-source models like GPT-OSS-120B and DeepSeek-V3.1 showed smaller, more consistent shifts across all tasks.
An interesting observation was the partial misalignment between stated beliefs and behaviors. While belief shifts were often reflected in actions, the magnitudes sometimes differed. This suggests that LMs might express a change in belief without fully acting on it, or vice versa, highlighting the complexity of LM cognition.
The length of accumulated context also played a role. In intentional tasks, stated belief changes appeared early in conversations, but behavioral changes grew substantially with longer interactions. For non-intentional reading, longer content generally led to greater belief shifts for conservative topics, while for progressive topics, shifts often emerged early and then stabilized.
Crucially, the study found that belief shifts are not solely driven by access to specific topic-relevant information. Even when highly relevant sentences were masked or concatenated, shifts still occurred, suggesting that the broader contextual framing of the entire reading material contributes significantly to these changes. This implies that even seemingly benign interactions can subtly alter an LM’s judgment over time.
This research uncovers a fundamental risk for the reliability of LM assistants in long-term, real-world applications. As users increasingly trust and rely on LMs, the hidden accumulation of context and the resulting belief drift could lead to inconsistent experiences and unpredictable behaviors. Understanding these dynamics is crucial for developing more robust and trustworthy AI systems. For more details, you can read the full paper here.
