TLDR: The research introduces “Brain-like Space,” a unified geometric framework to compare how AI models organize information internally with human brain networks. By mapping the spatial attention patterns of 151 Transformer-based models onto seven canonical brain networks, the study reveals an arc-shaped distribution reflecting varying degrees of “brain-likeness.” Key factors influencing this brain-likeness are pretraining paradigms (data augmentation, training objectives, distillation) and positional encoding schemes, which either promote global semantic abstraction and deep fusion or focus on local details. Interestingly, a model’s brain-likeness does not always directly correlate with its performance on specific tasks, suggesting that brain-like organization is an “expressive preference” rather than a direct measure of task superiority.
For decades, scientists have pursued a grand ambition: to understand intelligence and build it. Modern artificial neural networks now excel in tasks like language, perception, and reasoning, often rivaling human capabilities. Yet, a fundamental question remains – do these advanced AI systems organize information in a way that mirrors the human brain?
Traditional studies comparing AI and the brain have often been limited. They typically rely on showing both systems the same inputs and tasks, making it difficult to compare models with different modalities, such as a large language model (LLM) versus a large vision model (LVM). There hasn’t been a common framework to understand their intrinsic organizational logic.
Introducing the Brain-like Space
A groundbreaking new concept, the “Brain-like Space,” has been introduced to overcome this challenge. This unified geometric space allows any AI model to be precisely situated and compared by mapping its internal spatial attention patterns onto canonical human functional brain networks. Crucially, this comparison is independent of the model’s input type, task, or sensory domain.
The Brain-like Space is constructed by first identifying seven key functional networks in the human brain (such as visual, somatomotor, and default mode networks) using resting-state fMRI data. Then, for each attention head within a Transformer-based AI model, a spatial attention graph is created. Five graph-theoretic metrics are extracted from both the brain networks and the AI model’s attention heads. The similarity between these metrics forms a seven-dimensional vector, placing each attention head within the Brain-like Space. When projected into two dimensions, this space reveals a clear, arc-shaped geometry.
Key Discoveries in the Brain-like Space
An extensive analysis of 151 Transformer-based models, including state-of-the-art large vision models (LVMs), large language models (LLMs), and large multimodal models (LMMs), uncovered fascinating patterns:
- The models distribute themselves along a continuous arc within this space, reflecting a gradual increase in “brain-likeness.”
- Different models exhibit distinct distribution patterns, influenced not just by their modality (vision, language, or multimodal) but also by their pretraining paradigms and positional encoding schemes.
How Pretraining Shapes Brain-likeness
The study highlights that the way an AI model is pretrained acts as a “meta-regulator” for its brain-like organization:
-
Data Augmentation: Strategies that emphasize global disruption and viewpoint changes, like Mixup and RandAugment, encourage models to learn abstract, context-aware attention patterns. These models tend to show higher brain-likeness, matching more with higher-order cognitive brain networks. In contrast, augmentations focusing on local stability (e.g., grayscaling, Gaussian blur) lead to less brain-like organization, primarily matching with basic visual and limbic networks.
-
Training Objectives: Self-supervised learning objectives that promote semantic abstraction, such as those in DINO and BEiT, significantly increase brain-likeness. These objectives compel models to capture stable, global semantic relationships. Conversely, objectives focused on pixel-level reconstruction of masked regions, like in MAE, prioritize fine-grained detail restoration at the expense of global abstraction, resulting in lower brain-likeness.
-
Distillation Strategy: Distilling knowledge from CNN-based teachers into Transformer models (e.g., DeiT) can suppress the emergence of brain-like organization. CNNs introduce local inductive biases, which can overwrite the Transformer’s native global attention mechanisms, leading to less brain-like structures. Interestingly, this suppressive effect becomes more pronounced in larger models.
Positional Encoding and Cross-Modal Fusion
The study also examined the role of positional encoding schemes, particularly in language and multimodal models:
-
Language Models: Most large language models, whether using Rotary Positional Encoding (RoPE) or other schemes like in GPT-2 and BERT, tend to concentrate in the most brain-like regions of the space, indicating a strong organizational similarity to brain networks.
-
Multimodal Models: For models like CLIP and BLIP that use learnable positional encodings, their vision and language components show a clear “division of labor.” The language component is highly brain-like, while the vision component is less so, focusing on low-level structural features. However, RoPE-based multimodal models facilitate “deep fusion” between modalities. RoPE provides a unified geometric prior, allowing vision and language components to become more integrated and both exhibit higher brain-likeness.
Brain-likeness vs. Task Performance: Not Identical Twins
One of the most intriguing findings is that a model’s degree of brain-likeness does not always directly correlate with its performance on downstream tasks, such as image classification accuracy. While a general positive trend was observed, it was not statistically significant across all model families. This suggests that models might prioritize computational efficiency or task-specific robustness over replicating brain-like structures. Brain-likeness should be viewed as an independent metric, an “expressive preference,” offering insights into a model’s organizational principles rather than a direct measure of its superiority in specific tasks.
Also Read:
- Decoding the Emotional Core of Large Language Models
- Dynamic Plasticity Enhances Transformer Learning and Adaptation
A Path Towards a Unified Science of Intelligence
This research provides the first unified framework for quantifying, visualizing, and comparing intelligence across domains, revealing deep organizational principles that bridge machines and the brain. It suggests that the emergence of brain-likeness in AI is an evolutionary outcome driven by biologically compatible optimization objectives, sufficient representational freedom, and effective fusion across different modalities. This work opens a promising direction toward a unified science of intelligence that transcends the boundaries between machines and the brain. You can read the full research paper here.
