TLDR: This research introduces a novel LLM-based simulation framework to model the evolution of US citizens’ attitudes towards China over two decades (2005-2025). By integrating real-world media data, detailed user profiles, and a cognitive reflection mechanism, the framework successfully reproduces historical opinion trends. The study also explores interventions like debiased news exposure and a “devil’s advocate” agent, demonstrating their effectiveness in mitigating negative attitude shifts. Findings reveal how media framing and specific news categories influence public perception, offering insights into international bias formation and promoting media literacy.
A recent research paper explores a novel approach to understanding how public opinion evolves, specifically focusing on the attitudes of US citizens towards China. This study leverages the power of Large Language Models (LLMs) to simulate complex, large-scale human cognitive trends, a significant step in the field of opinion evolution modeling.
Unlike previous works that often focused on isolated events or opinions within a single country, this research is groundbreaking as it models the large-scale attitude evolution of an entire population – US citizens – towards another country, China. The researchers developed a comprehensive framework to tackle the complexities of this broad scenario. This framework integrates several key components: collecting real-world media data, creating detailed user profiles for simulated agents, and designing a cognitive architecture that allows for realistic opinion updates.
The simulation successfully reproduced the actual trend of US attitudes towards China over a 20-year period, from 2005 to the present day. A crucial aspect of their methodology involves using LLMs to introduce “debiased media exposure.” This means extracting neutral events from news content that is typically subjective, helping to uncover the underlying reasons for polarized opinion formation. Furthermore, the study introduced a “devil’s advocate” agent, designed to explain rare instances where negative attitudes towards China reversed to positive ones, correlating with shifts in how Americans access information about the country.
The findings not only validate the proposed framework but also highlight the significant impact of biased framing and selection bias in shaping public attitudes. The work contributes to a new way of using LLMs for modeling cognitive behaviors in a large-scale, long-term, and cross-border social context. It offers valuable insights into how international biases are formed and provides implications for media consumers to better understand the factors influencing their perspectives, ultimately supporting the broader societal goal of bias reduction and cross-cultural tolerance.
The study builds upon existing research in opinion dynamics and LLM-based social simulation. It draws inspiration from theories like selective exposure, social identity, and cognitive dissonance. Previous LLM simulations have explored various phenomena, from small village interactions to large-scale social behaviors like polarization and misinformation spread. However, this paper uniquely combines cognitive mechanisms with data-driven real-world macro-trends in international perceptions.
To create a realistic simulation, the researchers gathered extensive preliminary data. For agent initialization, they used two public datasets: X/Twitter data (3,849 profiles) and the 2024 General Social Survey (3,309 respondents). These datasets were processed and merged to generate approximately 2000 comprehensive agent profiles, each with 50 features including demographics, political preferences, media habits, and specific views on US domestic issues. This meticulous approach ensured the agent pool was a representative sample of the US population.
For news data, over 100,000 articles about China were collected from 56 publications over the 20-year period, primarily from mainstream US and UK news sources. This vast dataset was crucial for modeling how media influences international attitudes. The ground truth attitudes of US citizens towards China were obtained from the Pew Research Institute and Gallup, Inc., providing a reliable benchmark for comparison.
The simulation framework operates through an iterative cycle. Each year, a distribution engine samples news articles and broadcasts only their headlines to the agents. Agents then select articles to read based on their individual profiles, including demographics, political views, interests, and media habits. Once selected, the full text of these articles is retrieved and sent to the agents for the reflection process.
The reflection mechanism is central to the framework, designed around cognitive dissonance theory. When agents encounter new information, they decide if it contradicts their existing beliefs. If a contradiction exists, agents can choose to “revise” their prior view, “reinforce” their existing opinion by rationalizing it, or “dismiss” the importance of the conflicting belief. This process allows for a nuanced and human-like update of opinions. After reflection, new cognitions are broken down into specific topics, and agents update their views on these topics, quantifying their perception of China on a scale from -2 to 2.
At the end of each simulated year, agents participate in an overall attitude survey, similar to the real Pew survey, to assess their current opinion of China on a scale from 1 (Very unfavorable) to 4 (Very favorable). This aggregated data is then compared against the ground truth to evaluate the simulation’s accuracy.
The experiments involved 100 agents, each reading and reflecting on 10 news articles per year. The main simulation results showed a strong correlation with real-world trends, including the initial slight negativity, an inflection point around 2010, and an increasingly negative trend thereafter. While the simulation largely captured the trends, it did not fully reproduce sudden increases in positivity observed in 2017 and 2024-2025, suggesting factors beyond simple media exposure might be at play.
To address these “reversals,” two intervention mechanisms were implemented: “Debiased News Exposure” and a “Devil’s Advocate” agent. The debiased news mechanism aimed to simulate the effect of opinions being informed by neutral events rather than subjective news. The devil’s advocate agent, on the other hand, critiqued potential weak points in reporting, offering alternative perspectives. Both interventions successfully mitigated the negative trend, suggesting that being exposed to more neutral information or critically analyzing narratives can alleviate negative opinions, mirroring real-world attitude shifts.
Ablation studies further underscored the importance of each component of the framework. Removing the cognitive mechanism led to agents quickly adopting negative views, emphasizing the need for a nuanced cognitive design. The absence of detailed user profiles also resulted in a similar negative trend, indicating that individual identities help maintain more stable views. Interestingly, removing the article selection process slowed the descent into negativity, suggesting the presence of confirmation bias where agents tend to select articles aligning with existing views. Finally, using only headlines instead of full article text also resulted in slower negative trends, highlighting the necessity of rich, abundant media text for accurate modeling of macro-trends.
Beyond overall trends, the study also analyzed influencing factors. A domain analysis revealed that news articles covering technology, lifestyle, sports, and entertainment generally led to positive views towards China. Conversely, articles on economics, politics, and health strongly contributed to negative views. These findings offer insights into how media framing influences public perception and potential avenues for fostering more balanced perspectives.
Also Read:
- Building Smarter AI Agents: A New Framework for Realistic Social Simulations
- Unveiling and Correcting Biases in Large Language Models with BiasGym
In conclusion, this research presents a robust simulation framework that accurately models long-term attitude changes in a population, specifically US citizens’ perceptions of China. The framework’s success, with an overall Mean Absolute Error (MAE) of 8.3% compared to ground truth, demonstrates the potential of LLMs in understanding complex social dynamics. The interventions and factor analyses provide valuable insights for both media consumers and publishers, encouraging a more critical approach to news consumption and a push towards more objective reporting. For more details, you can read the full research paper here.
