TLDR: This article synthesizes findings from the CHI 2025 Tools for Thought Workshop, exploring how Generative AI (GenAI) impacts human cognition. It details both the risks, such as reduced critical thinking, hindered learning, and homogenized creativity, and the opportunities for augmenting human thought through various AI-driven approaches like provocation, task structuring, information transformation, and emotional augmentation. The paper also addresses the evolving nature of human-AI interaction, the importance of understanding workflows and expertise, and the critical need for robust research and evaluation methods to protect and enhance human cognitive abilities in the age of AI.
The rapid advancement of Generative AI (GenAI) is fundamentally reshaping how humans think, learn, and create. A recent synthesis from the CHI 2025 Tools for Thought Workshop, titled “Understanding, Protecting, and Augmenting Human Cognition with Generative AI,” brings together insights from 56 researchers and designers to map out the opportunities and risks this transformation presents for human cognition.
Authored by a collaborative team including Lev Tankelevitch, Elena L. Glassman, Jessica He, and others from leading institutions like Microsoft Research, Harvard University, and Carnegie Mellon University, this paper highlights the urgent need to understand how GenAI affects our minds, from critical thinking to memory and creativity. The workshop aimed to bridge the emerging science of AI’s cognitive impact with practical design strategies for building tools that both safeguard and enhance human thought.
Understanding AI’s Impact on Human Cognition
GenAI systems are designed to automate many cognitive tasks, such as writing and research, which are traditionally how people learn and build expertise. This shift means human workflows are moving from direct production to ‘critical integration’ of AI-generated material. This change can manifest in several ways:
- Critical Thinking: AI might discourage critical thinking by making information consumption more passive. Systems that agree with users or homogenize outputs can create ‘echo chamber’ effects. Over-reliance on AI, especially when self-confidence is low or AI confidence is high, can lead to accepting inaccurate information. The polished nature of AI outputs can also minimize the ‘state of perplexity’ necessary for reflective thinking, creating an ‘illusion of comprehensive understanding’.
- Learning: Students and novices, who are still developing self-regulation and mental models, may be particularly vulnerable. They might use AI in less intentional ways, leading to unproductive paths and over-reliance. Offloading lower-level understanding to AI can impede the development of crucial metacognitive strategies and schemas, as students bypass the cognitive processes involved in deliberate learning.
- Creativity: GenAI’s ability to produce high-fidelity outputs quickly risks shifting designers’ focus from core problem framing to merely tweaking polished results, potentially leading to design fixation and homogenization of creative outputs. However, AI can also serve as an ‘instrument of inquiry,’ shaping perceptions, conceptions, and the manipulation of ideas, especially for designers who understand its limitations.
Contextual Factors: Workflows, Expertise, and Human Values
Beyond individual cognitive domains, GenAI impacts workflows, expertise, and human values like motivation and well-being. The concept of ‘cognitive laziness’ versus ‘resource reallocation’ is explored, questioning when offloading tasks to AI is beneficial for deploying cognitive energy elsewhere. The paper emphasizes that understanding complex, non-linear workflows is key. For instance, in programming, AI might handle code generation, while humans focus on design specification and refactoring.
Expertise plays a crucial role; experts are better equipped to specify goals, prompt AI, and validate outputs. However, there’s a provocation: could optimal offloading to GenAI be an emerging ‘meta-expertise’? The paper also raises concerns about AI’s focus on efficiency potentially reducing space for ‘deep work’ and impacting intrinsic motivation and psychological well-being, as epistemic labor is vital for human needs like autonomy and competence.
Augmenting Human Cognition with AI
The workshop also explored how AI can actively augment human thinking. Approaches vary in their directness:
- Provoking Thinking: Some designs use AI as a ‘provocateur’ or ‘ignorant co-learner’ to challenge users’ thinking, prompt reflection, and foster critical thinking. This involves asking reflective questions, offering conflicting perspectives, or highlighting uncertainties.
- Structuring and Supporting Tasks: AI can provide scaffolding for processes like sense-making and ideation. Large Language Models (LLMs) can identify patterns, cluster examples, and generate analogical cues to help users discover novel applications.
- Transforming Information Representations: AI can dynamically transform information across modalities (e.g., speech to text, text to visuals) or levels of formality (e.g., formal code to informal sketches), expanding conceptual environments for users.
- Augmenting Emotions and ‘System 1’ Processes: Beyond conscious reflection, AI can augment ‘System 1’ thinking (emotions, heuristics). This could involve amplifying AI inaccuracies to foster creative risk-taking or personalizing learning experiences through emotional pathways to increase motivation.
Interacting and Interfacing with AI
The way humans interact with AI is also evolving. Beyond traditional conversational prompting, direct manipulation of on-screen objects can reduce cognitive load and express intent more accurately. ‘Feedforward’ information about prospective AI output can help users disambiguate prompts and reflect on their choices. The idea of ‘full-duplex communication,’ where AI continuously generates and proactively communicates ‘thoughts,’ could lead to a shared cognitive space.
However, these new interaction paradigms also raise questions. Replacing text prompting with direct manipulation might remove opportunities for users to think through their intents. The blending of human and AI-generated content could also blend biases. Future work needs to map the cognitive affordances of different modalities and how they express or transform knowledge.
Also Read:
- Integrating Artificial Intelligence and Cognitive Science: A Path to Deeper Understanding
- Interactive Virtual Personas: A New Frontier for User-Centered Design
Augmenting Workflows and Future Research
True cognitive augmentation requires thinking beyond discrete tasks to broader workflows. This could lead to ‘activity-centric computing,’ where application boundaries dissolve to support overarching goals. AI can also support collaborative workflows, potentially mitigating over-reliance seen in individual interactions. However, challenges remain in integrating AI seamlessly into complex, multi-person workflows and managing autonomous ‘agentic’ AI systems.
The paper concludes by emphasizing the need for formative research on GenAI users, their workflows, and expertise. It calls for a coherent approach to rigorous research, including new theories, construct definitions, and measurement methods, especially for qualitative, open-ended tasks. Longitudinal studies are crucial to understand the long-term impacts of GenAI on human cognition. The full research paper can be found here.
