TLDR: A new research paper introduces CoRE, a large-scale benchmark, to evaluate how Large Language Models (LLMs) reason about emotions using cognitive appraisal theory. The study reveals that while LLMs exhibit human-like cognitive structures for emotions, they also show diverse reasoning patterns and biases, particularly with nuanced emotions, suggesting current AI training methods may be insufficient for truly coherent emotional understanding.
In the rapidly evolving landscape of Artificial Intelligence (AI), a crucial question emerges: can machines truly understand and reason about emotions? While many Large Language Models (LLMs) have been trained to recognize or generate emotion-related text, a recent research paper delves deeper, exploring whether these advanced AI systems can reason about emotions through the lens of cognitive appraisal theory.
The paper, titled “Do Machines Think Emotionally? Cognitive Appraisal Analysis of Large Language Models,” by Sree Bhattacharyya, Lucas Craig, Tharun Dilliraj, Jia Li, and James Z. Wang, moves beyond simple emotion recognition. It investigates the internal cognitive structures LLMs implicitly use for emotional reasoning, drawing heavily from cognitive appraisal theory – a foundational framework in psychology that explains how emotions are triggered and differentiated based on an individual’s interpretation of a situation.
Understanding Cognitive Appraisal
Cognitive appraisal theory suggests that our emotional experiences are shaped by how we evaluate different aspects of a situation, such as whether it aligns with our goals, who is responsible, or how pleasant it is. For example, feeling pride might involve appraising a situation as goal-congruent and attributing success to oneself, while anger might stem from appraising a situation as unfair and caused by someone else. The researchers aimed to see if LLMs exhibit similar patterns of reasoning.
Introducing CoRE: A New Benchmark
To conduct this in-depth evaluation, the researchers introduced a new, large-scale benchmark called CoRE (Cognitive Reasoning for Emotions). This benchmark is designed to assess LLMs’ cognitive reasoning in emotion-laden scenarios. It covers 15 distinct emotion categories (like Happiness, Fear, Anger, Guilt, and Surprise) and 16 fine-grained cognitive appraisal dimensions (such as pleasantness, effort, control, and certainty). CoRE comprises nearly 5,000 prompts and over 34,000 model-generated appraisals across seven different LLMs, including both open-source and proprietary models like DeepSeek R1, GPT-o4-mini, Gemini 2.5 Flash, LLaMA 3, Phi 4, Qwen 3, and Qwen QwQ.
Unlike previous studies that focused on LLMs predicting emotions or appraisals for *other* agents, this research uniquely focuses on *self-appraisals* – how LLMs reason about their *own* emotional responses in hypothetical situations. This approach provides a more direct window into the models’ internal emotion representations.
What Drives Emotion in LLMs?
The study employed two main methods to understand LLMs’ implicit appraisal strategies. First, an unsupervised analysis (Principal Component Analysis) revealed latent cognitive dimensions. It found that for most LLMs, similar to humans, “valence” (pleasantness/unpleasantness) was the most dominant factor. However, LLMs often prioritized “effort” and “obstacle” features more prominently than humans. The concept of “agency” (internal vs. external control/responsibility) also emerged as a strong latent dimension, with models showing plausible associations between fairness and responsibility.
Second, a regression-based analysis identified which cognitive dimensions best predict specific emotion classes. The findings largely aligned with established human cognitive appraisal theories. For instance, Happiness was consistently predicted by high enjoyment and absence of obstacles. Fear was strongly associated with high effort, suggesting fearful situations are appraised as demanding significant mental or physical exertion. Interestingly, for Anger, perceptions of unfairness (legitimacy-cheated) were the dominant factor across all models, rather than just general unpleasantness. This suggests LLMs might view anger as contingent on perceived injustice, similar to human moral judgments.
How Emotions Are Represented Within and Across Models
The research also explored how emotions are represented within individual LLMs and how consistent these representations are across different models. Within models, a clear valence-based separation was observed, meaning positive emotions clustered together, and negative emotions clustered together. Emotions like anger and frustration were consistently close, reflecting shared appraisals of high agency and obstacles. Guilt, shame, and sadness also grouped tightly, indicating a low-control, self-evaluative profile.
However, the study found that models often struggled with more ambiguous emotions like hope, interest, or surprise, revealing gaps in fine-grained affective reasoning. For example, Gemini 2.5 Flash sometimes conflated appraisal spaces and showed identical representations for Hope and Interest.
When comparing emotion representations *across* models, the researchers found no universal appraisal distribution. While LLMs share a broad emotional structure, their fine-grained appraisals for specific emotions diverged significantly. Guilt showed the highest agreement across models, indicating a stable appraisal profile, while Surprise, Sadness, Interest, and Challenge exhibited low agreement and high variability. This suggests that while LLMs can capture general emotional topologies, the nuances of how they represent specific emotions vary considerably, limiting the direct transferability of appraisal-based emotion modeling between them.
Also Read:
- Neural Networks Redefine the Debate on Symbolic Thought
- The Hidden Truth: LLMs Deceive Even Without Prompts
Implications for the Future of AI
The findings of this paper, available at https://arxiv.org/pdf/2508.05880, highlight that while LLMs exhibit broadly human-like cognitive appraisal structures, they often struggle with nuanced emotions and show inconsistencies or biases across abstract cognitive dimensions. For example, some models consistently interpret all emotions as inherently uncertain, or construe anger primarily through fairness appraisals. These patterns point to implicit biases in their internal emotional representations.
The researchers conclude by raising important questions about whether current AI training paradigms, such as next-token prediction or supervised fine-tuning, are sufficient for instilling truly coherent world models of emotion in LLMs. This benchmark and its analyses lay a crucial foundation for future research in developing more cognitively grounded and emotionally intelligent AI systems.
