TLDR: This paper introduces BRAIN, a novel continual learning approach that addresses the problem of shifting and biased brain signals in Vision-Brain Understanding (VBU) models, caused by human memory decay over long data collection periods. BRAIN uses De-bias Contrastive Learning to mitigate signal bias and Angular-based Forgetting Mitigation to prevent catastrophic forgetting, achieving state-of-the-art performance in decoding visual information from fMRI data.
The human brain is an incredibly complex system, with a significant portion dedicated to processing visual information, allowing us to recognize objects and understand scenes. Researchers often use techniques like functional Magnetic Resonance Imaging (fMRI) to study this intricate connection between vision and brain function. However, a new research paper highlights a critical challenge in this field: the natural process of memory decay in humans.
Over extended periods of data collection, such as the months or even years required for large-scale datasets like the Natural Scenes Dataset (NSD), participants may encounter visual stimuli they have seen before. As human memory naturally decays, individuals might struggle to recall previously seen images, leading to lower confidence in their responses. This decline in confidence results in fMRI signals from later sessions being weaker, more uncertain, and containing less visual context compared to earlier recordings. This inconsistency creates a compounding bias in brain signal representations, making it difficult for Vision-Brain Understanding (VBU) models to learn effectively and ultimately degrading their performance.
Previous studies have largely overlooked this problem, treating all data points as equally reliable regardless of when they were recorded. This paper, titled BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding, is one of the first to specifically address this significant bias problem. The authors, Xuan-Bac Nguyen, Thanh-Dat Truong, Pawan Sinha, and Khoa Luu, propose a novel solution called BRAIN (Bias-Mitigation Continual Learning).
The BRAIN approach is designed to train VBU models in a continual learning setup, allowing them to adapt to the gradual changes and mitigate the growing bias from each learning step. This aligns with the evolving nature of human memory and perception, offering a more biologically plausible solution than static training methods. It also enables incremental model updates as data is collected, accelerating research progress.
Two key innovations are introduced within the BRAIN framework:
De-bias Contrastive Learning (DCL)
To tackle the signal bias, the researchers developed De-bias Contrastive Learning. This new loss function models the bias factor by observing that participant response accuracy decreases over time. Essentially, DCL assigns a higher weight to the loss function for data collected when participants are experiencing more uncertainty or lack of confidence. This exponential weighting helps the model to account for and mitigate the increasing bias in later sessions, leading to more accurate representations.
Also Read:
- Advancing Brain-Computer Interfaces: New Methods for Motor Imagery EEG Classification
- The Collective Mind: How Shared Language Shapes Memory and Attention
Angular-based Forgetting Mitigation (AFM)
Continual learning models often suffer from “catastrophic forgetting,” where they lose knowledge learned from previous sessions when adapting to new data. To prevent this, BRAIN introduces the Angular-based Forgetting Mitigation approach. Unlike traditional methods that use Euclidean distance to measure feature discrepancies, AFM employs an angular metric for knowledge distillation. This means the model focuses on maintaining the direction of learned features rather than their absolute magnitudes. This approach is more robust to variations and shifts in feature representations that occur during continual learning, allowing the model to adapt to new data while preserving old knowledge.
The empirical experiments conducted on the Natural Scenes Dataset (NSD) demonstrate that the BRAIN approach achieves State-of-the-Art (SOTA) performance across various benchmarks. It consistently surpasses prior continual learning methods, such as Learning without Forgetting (LwF) and PLOP, and even outperforms non-continual learning methods where all data is combined and trained at once. The ablation studies further confirm the effectiveness of both DCL and AFM in improving performance and mitigating bias and forgetting.
In conclusion, the BRAIN framework offers a robust and effective solution to the challenge of inconsistent brain signals caused by memory decay in Vision-Brain Understanding. By addressing bias and catastrophic forgetting, this approach paves the way for more stable and generalizable learning from fMRI data, with significant implications for future applications like brain-image retrieval and reconstructing visual experiences.
