TLDR: A study found that the “reasoning effort” of advanced AI models, measured by tokens, directly correlates with human decision time on content moderation tasks. Both humans and AI expend more effort when faced with difficult, ambiguous choices, suggesting parallels in cognitive processing and offering new avenues for AI interpretability and safety.
The research paper “AI Reasoning Effort Mirrors Human Decision Time on Content Moderation Tasks” explores a fascinating parallel between how artificial intelligence models and human beings approach complex decision-making, particularly in the sensitive area of content moderation. This study, led by Thomas R. Davidson from Rutgers University–New Brunswick, sheds light on the internal workings of advanced AI and suggests that their “thinking” process can be surprisingly similar to our own.
At the heart of this research is the concept of “reasoning effort” in large language models (LLMs). Modern LLMs can now generate intermediate steps, often called “chain-of-thought” prompting, before arriving at a final answer. This process is akin to a human thinking step-by-step through a problem. The study compares this AI reasoning effort, measured by the number of “reasoning tokens” consumed, with the actual time humans take to make decisions on identical content moderation tasks.
The findings are compelling: across three leading frontier models—OpenAI’s o3, Google’s Gemini 2.5 Pro, and xAI’s Grok 4—there was a consistent positive association between the AI’s reasoning effort and human decision time. This means that when a task was more challenging for humans, requiring more thought and time, the AI models also expended greater computational effort, consuming more reasoning tokens. For instance, a standard deviation increase in AI reasoning effort was linked to over one second of additional decision time for human subjects.
The study used a paired conjoint experiment, presenting both humans and AI with synthetic social media posts and asking them to identify which post was more likely to violate a content moderation policy. A key observation was that both humans and models expended significantly more effort when important variables were held constant. For example, when two posts contained equally offensive terms, humans took, on average, 4.5 seconds longer to decide. Similarly, all three AI models consumed over one standard deviation more reasoning tokens in these scenarios. This suggests that when the choice isn’t obvious, both humans and AI switch from quick, intuitive judgments to more careful, deliberative analysis, a pattern consistent with dual-process theories of cognition.
Further descriptive analyses, visualized through heatmaps, reinforced these findings. Decisions required the most effort when terms in the posts were either identical or very similar in offensiveness. Humans took the longest when posts had no offensive terms or used anti-White language, and were fastest with clearly incongruous pairs, like homophobic terms alongside innocuous posts. The AI models showed similar patterns, with some model-specific nuances in which types of offensive language triggered the most reasoning.
A qualitative analysis of Gemini 2.5 Pro’s reasoning traces provided deeper insights. When both posts were equally offensive, the model explicitly acknowledged this and then shifted its focus to secondary contextual factors, such as the specific language used (prioritizing personal attacks over group statements), user identity, discussion topic, and engagement metrics. This mirrors how humans might delve into finer details when a clear-cut decision isn’t immediately apparent.
The implications of this research are significant. The observed parallels suggest that AI reasoning traces could be valuable for interpretability and decision-making in high-stakes applications like content moderation, law, medicine, and hiring. Longer reasoning traces could serve as a practical signal, indicating cases that warrant human review due to their inherent difficulty. While reasoning traces don’t always perfectly capture the AI’s actual decision-making process, their correlation with task difficulty offers a promising avenue for enhancing transparency and safety in AI systems. The study also highlights the need for further research into how different AI architectures, training data, and post-training procedures influence not just what models output, but how they “think.”
Also Read:
- Unpacking AI’s Inner Monologue: When Does Chain-of-Thought Truly Guide, and When Does It Justify?
- AI Models Outperform Individual Humans in Predicting Everyday Social Norms
This work was supported by a Foundational Integrity Research award from Meta and API credits from OpenAI’s Researcher Access Program. For more detailed information, you can refer to the full research paper: AI Reasoning Effort Mirrors Human Decision Time on Content Moderation Tasks.
