TLDR: A study analyzed 1,006 LLM papers citing psychology, revealing increasing interdisciplinary engagement since 2023, particularly with Neural Mechanisms and Psychometrics. While some psychology theories are widely adopted (e.g., Theory of Mind, Dual-Process Theory), many remain underexplored. The research also identifies common misapplications, such as conceptual overgeneralization and reliance on secondary citations, and proposes recommendations for more responsible and rigorous interdisciplinary practices to advance AI development.
The rapid evolution of Artificial Intelligence, particularly Large Language Models (LLMs), has sparked a growing interest in understanding these complex systems through the lens of human cognition. A recent study titled “The Incomplete Bridge: How AI Research (Mis)Engages with Psychology” delves into how AI research currently interacts with the field of psychology, highlighting both successful integrations and areas needing improvement.
Authored by Han Jiang, Pengda Wang, Xiaoyuan Yi, Xing Xie, and Ziang Xiao from Johns Hopkins University, Microsoft Research Asia, and Rice University, the paper explores the interdisciplinary synergy between AI and psychology. The researchers analyzed over 1,000 LLM-related papers published in premier AI venues between 2023 and 2025, along with the more than 2,500 psychology publications they cited. Their goal was to map out the patterns of integration, identify frequently referenced psychology domains, and pinpoint areas that remain underexplored or even misapplied.
Growing Interdisciplinary Engagement
The study reveals a clear trend: LLM research has increasingly cited psychology papers, especially since early 2023. This surge in interest coincides with the release of advanced models like GPT-3.5-Turbo and GPT-4, which prompted AI researchers to look into the internal workings of LLMs. Initially, citations focused on psychology areas closely related to model mechanisms and evaluation, such as Neural Mechanisms and Psychometrics & Judgment and Decision-Making (JDM). The increased accessibility of open-source models like Llama2 further facilitated this interdisciplinary collaboration.
Psychology research is broadly integrated across various LLM study areas. For instance, LLM research on educational applications frequently draws from educational psychology, while advanced reasoning tasks often reference cognitive neuroscience. More complex LLM areas, such as model adaptation and social intelligence, tend to cite a wider range of psychology topics, reflecting the multifaceted nature of these AI capabilities.
Prominent and Overlooked Psychological Theories
The research identifies several psychological theories and frameworks that are most commonly cited in LLM papers. Among the top ten are Dual-Process Theories, Theory of Mind, Heuristics and Biases Program, Executive Functions, and Classical Test Theory. For example, Dual-Process Theory, which describes human thought as arising from two distinct systems (intuitive and conscious), has inspired LLM researchers to moderate and steer model behaviors. Theory of Mind, the ability to understand others’ mental states, is frequently used to interpret and enhance LLMs’ social intelligence. Classical Test Theory, a psychometric framework for evaluating psychological tests, guides the design of evaluation methods for LLMs.
However, the study also points out that many valuable psychological theories remain underexplored. For instance, the Biopsychosocial Model (which views health as an interaction of biological, psychological, and social factors) could offer a holistic framework for modeling user behavior in LLMs. Critical Race Theory could provide a lens to assess and mitigate biases in model outputs. Similarly, theories like Predictive Coding, Hofstede’s Cultural Dimensions Theory, and Prospect Theory offer rich insights that could significantly advance LLM development in areas like adaptive communication, cultural sensitivity, and human-like decision-making under uncertainty.
Challenges and Misapplications
A significant finding of the study is the presence of common misapplications when LLM research references psychology. Using Theory of Mind as a case study, the authors identify four key issues:
- Conceptual Overgeneralization and Misclassification: Researchers often use broad terms like “ToM tasks” without distinguishing between different levels of mental-state reasoning (e.g., first-order vs. second-order false belief tasks), or they miscategorize related but distinct psychological processes as ToM.
- Partial or Incomplete Citation: There’s a tendency to cite a few well-known psychology papers while overlooking other equally important or more relevant but less “representative” works. This can lead to a narrow understanding of the field.
- Misinterpretation or Misrepresentation of Findings: Some LLM studies cite psychology papers inappropriately or selectively emphasize positive findings while ignoring limitations or ongoing debates within the original research.
- Secondary Citation Errors: Researchers often rely on interpretations of psychology theories by other AI/NLP researchers rather than directly consulting the primary psychology literature. This can amplify misunderstandings and oversimplifications.
Also Read:
- Assessing LLM Therapists: A New Approach to Understanding Client Exploration
- Navigating the Future of AI: A Comprehensive Look at Language Model Alignment and Safety
Towards Responsible Interdisciplinary Practice
To foster a more robust and sustainable bridge between AI and psychology, the paper offers several recommendations:
- Theoretical Accountability: Researchers should clearly explain the origins, assumptions, and boundaries of the psychology theories they use, and acknowledge competing perspectives.
- Construct Operationalization: When applying psychological concepts, researchers should use standardized measurement tools or clearly justify how custom tasks reflect the underlying psychological construct.
- Collaborative Parity: Interdisciplinary work should be built on mutual respect and co-creation, encouraging joint authorship and diverse analytical perspectives from the outset.
- Open Interdisciplinary Infrastructure: Developing shared resources like reusable datasets on psychological constructs, case templates for interdisciplinary research, and cross-referencing maps can lower barriers to collaboration and enhance research quality.
In conclusion, while the integration of psychology into AI research is growing, there’s a critical need for more rigorous, nuanced, and responsible engagement. By addressing these challenges, AI systems can be developed with a deeper understanding of human intelligence, leading to more effective and ethically sound technologies.
