TLDR: “Shapes of Cognition” introduces a new paradigm for building intelligent agents (LEIAs) that mimic human-like cognitive processes. It proposes that intelligence relies on “shapes” – remembered constellations of knowledge (sensory, linguistic, conceptual, episodic, procedural) – to efficiently process information, recognize patterns, and minimize cognitive load. This framework, implemented within the HARMONIC architecture for OntoAgentic AI, applies to various aspects of agent function, including how they store knowledge (lexicon, ontology, episodic memory) and how they reason (language understanding, language generation, script learning). By focusing on typical cases and structured knowledge, this approach aims to create explainable, extensible, and trustworthy AI systems.
A new approach to understanding and building intelligent systems, called “Shapes of Cognition,” is emerging as a significant paradigm in computational cognitive modeling. This concept, developed by researchers at Rensselaer Polytechnic Institute, aims to create Language-Endowed Intelligent Agents (LEIAs) that can navigate the complexities of real life much like humans do: by recognizing patterns, acting on habits, reasoning by analogy, and generally minimizing mental effort.
At its core, a “shape” is a remembered collection of different types of knowledge – sensory, linguistic, conceptual, episodic, and procedural. These shapes allow agents to anticipate typical situations and respond efficiently. When unexpected events occur, shapes-based recovery methods, such as learning on the fly or seeking human help, guide the agent toward an actionable understanding. This framework is not vague; it involves specific objectives, hypotheses, modeling strategies, and knowledge bases, all implemented within a particular cognitive architecture.
The HARMONIC Architecture and OntoAgentic AI
The Shapes of Cognition paradigm grew out of work on LEIAs, which are neurosymbolic, multimodal, cognitive-robotic systems built on the HARMONIC architecture. This architecture features a cognitive layer that uses knowledge-based methods for reliability and transparency, and a robotic layer that employs machine learning for tactical capabilities.
LEIAs are central to a novel approach called OntoAgentic AI. Unlike typical agentic AI systems that rely heavily on large language models (LLMs) for orchestration, OntoAgentic AI uses a LEIA as the primary orchestrator. This allows for reliable, explainable control over the entire system and individual LEIA operations.
Meaning and Ontology
LEIAs process information based on meaning, which is defined by an unambiguous, language-independent ontology. This means that when LEIAs interpret, reason, or plan, they do so using meaning representations grounded in this ontology. These frame-based meaning representations are themselves a prime example of a “shape.” This ontological grounding offers several benefits: agents can leverage extensive knowledge about concepts, their reasoning processes are transparent and human-inspectable, and the core knowledge is language-independent, making it easier to adapt LEIAs to different natural languages.
Shapes in Knowledge Bases
The concept of shapes extends across various knowledge bases within LEIAs:
- Lexicon: Lexical shapes, part of “construction semantics,” precisely define the syntactic and semantic expectations of word senses. This helps agents learn new words, speeds up human lexical acquisition, and allows for more efficient development and evaluation of natural language understanding and generation systems. For instance, different phrases like “John admires his uncle,” “John looks up to his uncle,” and “John puts his uncle on a pedestal” can share the same semantic shape (an event with an agent and a theme) despite having different syntactic structures.
- Ontology: Frames, facets, inheritance hierarchies, and scripts are all ways shapes manifest in the ontology. Frames organize knowledge about types (e.g., the concept of SURGERY), while facets capture different strengths of constraints (e.g., a surgeon typically performs surgery in an operating room, but a regular person might perform a lifesaving procedure anywhere). Scripts, which are sequences of events, record how complex actions typically unfold, enabling agents to plan and recognize others’ plans.
- Episodic Memory: Episodic knowledge, which records instances of ontological concepts, also uses frame-based structures, mirroring the shapes of the concepts they instantiate. This allows for reasoning by analogy (reusing past successful plans), memory consolidation (identifying recurring patterns to form habits), and planning that incorporates individual preferences (remembering how different human collaborators prefer tasks to be done).
Shapes in Agent Reasoning
Shapes are crucial for how LEIAs reason and interact with language:
- Language Understanding: The process of understanding language involves multiple stages of syntactic and semantic analysis. Shapes, particularly syntactic shapes (tree structures) and semantic shapes (meaning representations), guide how lexical senses are selected and how dynamic transformations are applied to interpret sentences, even when words are used in non-basic ways.
- Language Generation: When an agent needs to express something, shapes of meaning – variable-inclusive templates of ontological concepts – help translate meaning representations into sentences. The overall shape of the meaning dictates how individual components are expressed. For example, the phrase “the bicycle is blue” might be rendered differently depending on whether “blue” is the main idea or a modifier within a larger concept like “the blue bicycle is expensive.”
- Script Learning: Learning complex procedural knowledge (scripts) is broken down into many component tasks. Shapes help conceptualize the varying difficulty levels of these tasks. Different “shapes of learning” can be visualized, showing which aspects (e.g., understanding input, clarifying ambiguities, filling in missing information) are challenging in a given scenario. This allows for targeted development and evaluation of an agent’s learning capabilities.
Also Read:
- Bridging Minds: How Large Language Models Are Enhancing Cognitive Architectures
- Rethinking AI’s Core: Is the ‘Agent’ Concept Holding Back Next-Generation Intelligence?
Broader Applications and Future Directions
The paper highlights several other areas where shapes play a role, including coreference resolution, dialog management, clarification strategies, assessing actionability in uncertain situations, robotic control, and even in visualizing system processes and algorithms.
In essence, “Shapes of Cognition” is a specific, comprehensive approach to cognitive modeling that emphasizes focusing on typical cases, developing generic recovery strategies, visualizing models for clarity, and evaluating systems based on clear expectations. It offers a path to building useful, explainable, and trustworthy agent systems by drawing insights from how humans effectively navigate a complex world. You can read the full research paper here.
