TLDR: MIRAGE is a new AI framework inspired by the human brain’s dual-process cognitive system (hippocampus and prefrontal cortex). It combines a fast neural decomposer (System 1) with a deliberate schema engine (System 2) to achieve systematic compositional generalization. Tested on the SCAN benchmark, MIRAGE shows high accuracy and the ability to generalize to novel tasks by explicitly managing and applying reusable knowledge structures, demonstrating a promising path for AI to handle complex, novel combinations of known elements.
Artificial intelligence systems have made incredible strides, especially with large language models, but one persistent challenge remains: systematic compositional generalization. This refers to an AI’s ability to understand and create new combinations from existing building blocks, much like how humans can understand “jump twice” if they know “jump” and “twice.” This capacity is a hallmark of human intelligence, yet it has proven difficult for AI to replicate.
A new research paper, “A Neuroscience-Inspired Dual-Process Model of Compositional Generalization”, introduces a novel framework called MIRAGE (Meta-Inference with Rules and Abstractions from Generalized Experience) that draws inspiration directly from how the human brain achieves this flexibility. The human brain utilizes an interplay between the hippocampus (HPC) and the prefrontal cortex (PFC) for rapid encoding of experiences and consolidating them into reusable knowledge structures, known as schemas.
MIRAGE: A Dual-Process Approach
MIRAGE mirrors this brain functionality with two interacting modules:
1. The Meta-trained Transformer Neural Decomposer (System 1): This module acts like the brain’s intuitive, fast pattern recognition system (neocortical “System 1”). It’s trained on a wide variety of compositional grammars and performs one decomposition step at a time, iteratively refining the sequence representation.
2. The Schema Engine (System 2): Analogous to the HPC-PFC “System 2” loop, this component is responsible for deliberate reasoning. It dynamically extracts, ranks, and applies reusable schemas, storing temporary information in an episodic memory and expanding it as needed.
The brilliance of MIRAGE lies in how these two systems work together. System 2 identifies relevant schemas and their priorities, feeding this structured information into System 1’s context. The Transformer then uses this schema-augmented input to break down complex problems into smaller, manageable subproblems. After each processing step by System 1, System 2 manages the application of schemas and placeholder substitutions, continuously refining the problem’s representation for subsequent processing. This iterative, schema-guided refinement allows MIRAGE to systematically tackle complex compositional tasks.
Also Read:
- Neuro-Symbolic AI Learns to Simplify Complex Formulas with Enhanced Efficiency
- Knowledge Grafting: A New Method for Efficient AI on Resource-Limited Hardware
Impressive Performance and Key Insights
The researchers evaluated MIRAGE on the SCAN benchmark, a standard test for compositional generalization. MIRAGE achieved over 99% accuracy across all task splits, a significant improvement over traditional Transformers that often struggle with generalization on these tasks. What’s particularly noteworthy is that MIRAGE was trained in a task-agnostic manner on randomly generated grammars, not specifically on SCAN examples, demonstrating its strong generalizability.
Ablation studies, where specific components of MIRAGE were removed or altered, revealed critical insights into its success:
- Priority Tokens are Essential: Explicitly scheduling schema priorities is crucial. Without them, accuracy significantly drops, indicating that the model relies heavily on these cues to choose the correct schema when multiple options overlap.
- Iterative Refinement is Key: The step-by-step decomposition and iterative refinement process is vital for generalization. Attempts to train Transformers for single-shot, complete decomposition were largely unsuccessful.
- Bounded Depth Training: Training the model on sequences with a limited composition depth (e.g., 2-deep) proved more effective for generalization, even when applied to much deeper compositions during inference. This suggests a focused learning paradigm on prioritized pattern matching is more beneficial.
- Accurate Schema Libraries: Even minor errors in the extracted grammar or schema definitions drastically reduce performance, underscoring the necessity of high-quality schema extraction.
These findings strongly support the idea that combining an explicit schema extraction system with a fast neural processor is both necessary and sufficient for systematic compositional AI. MIRAGE represents a promising alternative to training massive, homogeneous neural networks for step-by-step inference, instead augmenting a smaller reasoner with an external, brain-inspired module. This approach, which separates knowledge from inference, could pave the way for more human-like intelligence in AI systems.
