TLDR: The paper proposes a ‘heterarchy’ model for the neocortex, where regions can operate both hierarchically and in parallel. It introduces the ‘Thousand Brains Theory,’ suggesting every cortical column is a sensorimotor system that learns complete 3D objects by integrating sensory and movement data. The theory reinterprets long-range connections, highlighting the thalamus’s crucial role in transforming sensory and motor information between egocentric and allocentric reference frames, and how hierarchical connections facilitate learning compositional object structures. This new perspective has significant implications for neuroscience and AI, advocating for sensorimotor integration and local learning in artificial systems.
For decades, our understanding of the brain’s neocortex has largely revolved around a hierarchical model, where sensory information ascends through layers, becoming progressively more complex. However, new research challenges this traditional view, proposing a more flexible organization termed ‘heterarchy’. This novel perspective suggests that brain regions can operate both hierarchically and in parallel, offering a fresh interpretation of how our brains process sensory and motor information.
The Thousand Brains Theory: A New Foundation
At the heart of this new theory is the ‘Thousand Brains Theory’, which posits that every cortical column in the neocortex functions as a sensorimotor learning system. This means that even primary sensory regions, traditionally thought to only process simple features, are capable of learning and recognizing complete 3D objects. Columns achieve this by integrating sensory input with information about the body’s movements, such as how a finger moves over an object or how eyes scan a scene. This integration allows the brain to build detailed, structured models of the world.
Rethinking Long-Range Connections
The paper delves into the various types of long-range connections within the neocortex and between the neocortex and the thalamus, assigning them specific roles within this heterarchical framework:
- Hierarchical Cortico-cortical Connections: These are the classic feedforward and feedback pathways. Instead of merely processing increasingly complex features, these connections are proposed to be crucial for learning ‘compositional structure’. This means understanding how larger objects are composed of smaller, previously learned objects (e.g., a bicycle made of wheels, frame, and pedals). Information flows both up (feedforward, identifying components) and down (feedback, predicting expected features and locations).
- Non-Hierarchical Cortico-cortical Connections: These connections link columns within the same brain region, between hemispheres, or even across different sensory modalities (like vision and touch). Their primary role is to enable rapid consensus among columns. If multiple columns are observing different parts of the same object, these connections allow them to quickly agree on the object’s identity, even with partial information. This ‘voting’ mechanism explains how we can recognize objects almost instantly, sometimes with a single glance or touch.
- Cortico-Thalamic Connections: The thalamus, often seen as a simple relay station, is given a central and dynamic role. Projections from the neocortex back to the thalamus are believed to modulate how thalamic relay cells respond to sensory input.
- Layer 5 Motor Connections: Every cortical column is thought to have a motor output. These connections send motor commands to sub-cortical areas, but also send a copy of these commands (an ‘efference copy’) to higher cortical regions via the thalamus. This highlights that motor processing is an intrinsic part of every cortical column, operating in both parallel and hierarchical ways.
The Thalamus as a ‘Pose Converter’
A significant proposal in this theory is the thalamus’s role in ‘pose conversion’. Our sensors (eyes, skin) move, and their orientation relative to an object constantly changes. The brain needs to compensate for these changes to maintain a stable perception of the world. The paper suggests that the thalamus transforms incoming sensory and movement data from an ‘egocentric’ (sensor-relative) orientation to an ‘allocentric’ (object-relative) orientation. Similarly, it converts allocentric motor commands from the cortex into egocentric commands for the body. This real-time transformation is critical for accurate object recognition and motor control.
Also Read:
- Monty: A Sensorimotor AI System for Agile and Efficient Learning
- Self-Supervised Learning Guides Robots to Bio-Inspired Movement
Implications for Artificial Intelligence
This new understanding of the neocortex has profound implications for artificial intelligence. Current deep learning models, while powerful, often struggle with issues like data efficiency, robustness, and understanding compositional structure. The principles outlined in this paper suggest a different path for AI development:
- Integrating sensor movement with sensed features at every level to learn structured models of complete objects.
- Using hierarchy to learn how objects are positioned relative to each other, mirroring the compositional structure of the real world.
- Emphasizing local learning and updates within general processing units (like cortical columns) rather than global gradient calculations and backpropagation across an entire network.
These insights could lead to more robust, efficient, and biologically plausible AI systems, potentially unifying AI with robotics by emphasizing the intimate relationship between intelligence and movement. For more details, you can read the full research paper here.
In conclusion, this research challenges the long-held hierarchical view of the neocortex, proposing a more dynamic ‘heterarchy’ where regions can operate both sequentially and in parallel. By reinterpreting the roles of long-range connections and highlighting the thalamus’s function as a ‘pose converter’, the paper offers a comprehensive theory of sensorimotor learning and compositional object recognition, with significant implications for both neuroscience and the future of artificial intelligence.
