TLDR: HiCL is a novel continual learning architecture inspired by the human hippocampus, designed to mitigate catastrophic forgetting in AI. It uses a dual-memory system with Dentate Gyrus (DG)-inspired sparse pattern separation for dynamic task routing, CA3-like pattern completion, and CA1-like integration with prioritized replay and Elastic Weight Consolidation. This biologically grounded approach enables efficient, adaptive learning across sequential tasks, achieving competitive accuracy at significantly lower computational costs by routing inputs to specialized experts without a separate gating network.
Artificial intelligence systems have achieved remarkable feats in various tasks, but they often struggle with a fundamental challenge known as catastrophic forgetting. This occurs when a neural network, trained sequentially on multiple tasks, tends to overwrite previously learned knowledge as it acquires new information. This limitation stands in stark contrast to the human brain, which continuously learns throughout life, integrating new skills while preserving old memories.
A new research paper introduces HiCL (Hippocampal-Inspired Continual Learning), a novel architecture designed to tackle catastrophic forgetting by drawing inspiration from the intricate circuitry of the human hippocampus. The hippocampus is a crucial brain region for memory formation, particularly in rapidly encoding new experiences and then consolidating them into long-term memory.
The Brain’s Blueprint for Learning
HiCL’s design is rooted in the hippocampal trisynaptic circuit, which involves three key subregions: the Dentate Gyrus (DG), CA3, and CA1. Each of these biological components has a specific role in memory processing, and HiCL translates these functions into computational modules:
- Dentate Gyrus (DG) – Pattern Separation: In the brain, the DG is responsible for pattern separation, ensuring that similar inputs are encoded as distinct, sparse representations. HiCL emulates this with a ‘sparse activation’ DG layer that enforces top-k sparsity, effectively orthogonalizing features. This helps in creating unique ‘feature vocabularies’ for different tasks.
- CA3 – Pattern Completion: The DG’s sparse codes feed into CA3, which acts as an autoassociative network for pattern completion, reconstructing full memory traces from partial cues. HiCL mirrors this with a lightweight multi-layer perceptron (MLP) that refines and transforms the DG outputs, functionally completing the pattern for further processing.
- CA1 – Integration: CA1 integrates the completed patterns from CA3 with direct cortical inputs, playing a role in memory consolidation. In HiCL, a CA1-inspired integration block combines the separated and completed signals, which then feeds into a consolidation stage that uses Elastic Weight Consolidation (EWC) and a prioritized replay buffer.
Dynamic Task Routing with DG-Gated Mixture-of-Experts
A core innovation of HiCL is its DG-gated Mixture-of-Experts (MoE) mechanism. Instead of relying on a separate, learned gating network, HiCL dynamically routes inputs to specialized ‘experts’ based on the similarity between their sparse DG representations and learned task-specific DG prototypes. Each expert maintains a prototype, an average of its DG codes, which acts as a compact summary of its specialized feature vocabulary. When a new input arrives, it’s processed by all experts’ DG modules, but only the expert whose feature vocabulary strongly resonates with the input will produce a strong activation, leading to accurate routing.
This biologically grounded gating strategy allows for differentiable and scalable task-routing, enhancing the model’s adaptability and efficiency in learning multiple sequential tasks. The sparse separation enforced by the DG layer ensures that each stored exemplar occupies a distinct sparse subspace, maximizing its utility during replay and minimizing interference between tasks.
Also Read:
- BRAIN: Learning from Evolving Brain Signals for Vision Understanding
- Adapting Anomaly Detection for Dynamic Medical Imaging
Dual-Memory Training and Efficiency
HiCL employs a two-phase training schedule: Phase I focuses on specialized encoding, forming and updating task prototypes, and shaping DG gating. Phase II consolidates prior knowledge through contrastive alignment, mitigating interference and preventing degradation. The model also incorporates prioritized replay of stored patterns to reinforce essential past experiences, mirroring how hippocampal replay consolidates memories.
Evaluations on standard continual learning benchmarks like Split CIFAR-10 and Split Tiny-ImageNet demonstrate HiCL’s effectiveness. It achieves competitive accuracy, often near state-of-the-art results, while significantly reducing computational costs. For instance, on Split CIFAR-10, the small HiCL model requires only 16.3 MFLOPs for inference, a fraction of what many other methods demand. This efficiency stems from its conditional computation, where sparse DG codes select a single expert for activation per input, avoiding the need to execute all expert pipelines.
While HiCL shows great promise, it currently operates under the assumption of known task boundaries during training. Future work aims to enable unsupervised task discovery and adaptive sparsity. For more in-depth technical details, you can refer to the full research paper: HiCL: Hippocampal-Inspired Continual Learning.
HiCL represents a significant step forward in developing AI systems that can learn continuously and adaptively, much like the human brain. By bridging neuroscience and machine learning, this work inspires architectures that are not only high-performing but also interpretable, modular, and biologically grounded, paving the way for more robust and efficient AI in real-world applications like autonomous robots and edge computing devices.
