TLDR: A new study investigated the factors driving the similarity between AI computer vision models and the human brain. By systematically varying model size, training data quantity, and image type for DINOv3 models, researchers found that all three factors independently and interactively impact brain-model convergence. Larger models, more extensive training, and human-centric images lead to higher brain-like representations. Crucially, the emergence of these representations in AI models follows a chronological order that mirrors biological brain development, aligning with specific cortical properties like expansion, thickness, and myelination. This research offers insights into both AI development and the fundamental principles of human visual processing.
Artificial intelligence models, particularly those designed for computer vision, are increasingly developing internal representations that bear a striking resemblance to how the human brain processes visual information. However, the precise reasons behind this convergence have remained a mystery. A recent study, titled Disentangling the Factors of Convergence between Brains and Computer Vision Models, delves into this complex relationship, systematically investigating how different aspects of AI model design and training contribute to this brain-like behavior.
The research, conducted by a team including Joséphine Raugel, Marc Szafraniec, Huy V. Vo, Camille Couprie, Patrick Labatut, Piotr Bojanowski, Valentin Wyart, and Jean-Rémi King from Meta AI and Ecole Normale Supérieure – PSL Université, focused on a family of self-supervised vision transformers called DINOv3. By carefully varying three key factors – the model’s architecture size, the amount of training data, and the type of images used for training – the scientists aimed to isolate their individual and combined effects on brain-model similarity.
Measuring Brain-Like Representations
To assess how closely the AI models mirrored the human brain, the researchers compared the models’ responses to natural images with brain activity recorded using two advanced neuroimaging techniques: ultra-high field functional magnetic resonance imaging (fMRI) and magneto-encephalography (MEG). fMRI provides high spatial resolution, showing which brain areas are active, while MEG offers high temporal resolution, revealing the timing of brain activity.
Three complementary metrics were employed to quantify brain-model similarity:
-
Encoding Score: This measured the overall similarity of representations, essentially how well brain activity could be linearly predicted from the AI model’s activations.
-
Spatial Score: This assessed whether the hierarchical organization of the AI model’s layers matched the brain’s visual hierarchy. For instance, do the model’s early layers align with sensory regions like V1, and its later layers with higher-level areas like the prefrontal cortex?
-
Temporal Score: Using MEG data, this metric examined if the model’s layers aligned with the temporal dynamics of brain activity, meaning early model layers predict early brain responses and late layers predict later responses.
Key Findings: What Makes AI Models See Like Us?
The study revealed that all three factors – model size, training amount, and image type – independently and interactively influenced how brain-like the AI models became.
The Impact of Training
The emergence of brain-like representations followed a specific developmental trajectory during training. Models first aligned with the early representations of sensory cortices. Only with significantly more training data did they begin to align with the later and more complex representations found in the prefrontal cortex. This suggests a staged acquisition of visual processing capabilities, mirroring how the human brain develops.
The Role of Model Size
Larger DINOv3 models consistently achieved higher brain-similarity scores across all metrics. They also converged more quickly during training. Interestingly, the increase in brain-like representations due to larger model size was more pronounced in higher-level cortical areas, such as the prefrontal cortex, compared to early visual regions.
The Influence of Image Type
The type of training data proved crucial. Models trained on human-centric images (e.g., photos from public Instagram posts, street views) achieved significantly higher brain-similarity scores than those trained on satellite images or cellular microscopic images. While even non-human-centric data led to some convergence in early visual areas, suggesting universal low-level visual features, human-centric data was essential for a more comprehensive alignment with the brain.
Connecting AI Development to Brain Biology
Perhaps one of the most striking findings was the strong correlation between the AI models’ developmental trajectory and various structural and functional properties of the human cortex. The brain areas that the AI models aligned with last during their training were precisely those known for:
-
Greater cortical expansion during development.
-
Larger cortical thickness.
-
Slower intrinsic timescales (meaning they integrate information over longer periods).
-
Lower levels of myelination (myelin facilitates faster neuronal transmission).
These characteristics are typical of associative cortices, which mature slowly throughout the first two decades of human life. This suggests that the sequential acquisition of representations in artificial neural networks might spontaneously model some of the developmental trajectories of brain functions, offering a new computational framework to understand biological visual processing.
Also Read:
- Mimicking Human Vision: A Deep Network for Perceiving Hidden Shapes
- BRAIN: Learning from Evolving Brain Signals for Vision Understanding
Implications for AI and Neuroscience
This research provides a clearer understanding of the interplay between architecture and experience in shaping how artificial neural networks come to perceive the world in a human-like manner. It highlights that while the model’s architecture provides a potential, the training data is critical in realizing brain-like representations. The study’s findings not only advance our knowledge of AI but also offer a promising framework for using AI models as tools to investigate the organizing principles of biological vision in the human brain, shedding light on how humans themselves come to represent their visual world.
