TLDR: MATL-DC is a novel transfer learning framework for EEG-based emotion recognition that effectively identifies emotions in new, unseen subjects without requiring their data during training. It achieves this by decoupling features to separate individual differences from emotional commonalities, aggregating subject data into ‘superdomains’ using MMD, and employing adaptive prototypes for stable learning. A pairwise learning strategy enhances robustness against label noise. Validated on public datasets, MATL-DC demonstrates superior or comparable performance to models that do use target domain data, offering a practical solution for generalized emotion recognition.
Emotion recognition using electroencephalography (EEG) signals is a rapidly growing area, particularly for affective Brain-Computer Interfaces (aBCIs). The core challenge lies in accurately decoding EEG signals and aligning their features, especially when dealing with new individuals or ‘unseen targets’. Traditional transfer learning models often rely heavily on both source and target domain data, meaning they need to be retrained for each new subject, which is costly and can lead to inflated performance. Furthermore, individual differences in EEG signals and the presence of label noise during emotion experiments pose significant hurdles.
To address these critical issues, researchers have introduced a novel framework called MATL-DC: A Multi-domain Aggregation Transfer Learning Framework for EEG Emotion Recognition with Domain-Class Prototype under Unseen Targets. This innovative approach aims to provide a robust and generalized solution for emotion recognition without needing to see the target domain data during the training process.
How MATL-DC Works
The MATL-DC framework is built upon several key modules designed to tackle the complexities of EEG-based emotion recognition:
First, a **Feature Decoupling Module** is designed to separate different types of information within the shallow features of EEG signals. It distinguishes between ‘class-invariant domain features’ (which capture individual differences between subjects) and ‘domain-invariant class features’ (which represent common emotional characteristics across subjects). This separation is crucial because variations in domain features often lead to dispersed class feature distributions, making accurate emotion recognition difficult.
Next, the **Multi-domain Aggregation Mechanism** comes into play. Recognizing that EEG samples from different subjects might have underlying connections rather than being entirely distinct, this mechanism aggregates multiple subject-specific domains into a smaller number of ‘superdomains’. This is achieved by quantifying the discrepancies between domain feature distributions using a method called Maximum Mean Discrepancy (MMD). By forming superdomains, the framework reduces computational costs and enhances the model’s ability to generalize across subjects by leveraging shared features.
Within each superdomain, the framework employs an **Adaptive Prototype Updating** mechanism. This involves creating ‘domain prototype representations’ for each superdomain (representing the essential characteristics of domain features) and ‘class prototype representations’ for different emotional states within each superdomain. These prototypes are continuously updated during training, with an adaptive weight parameter that adjusts the update rate, ensuring stable and optimal learning.
Finally, to combat the unavoidable problem of label noise in EEG emotion experiments (where individuals might have different subjective feelings to the same stimulus), MATL-DC adopts a **Pairwise Learning Strategy**. Instead of treating classification as a problem of assigning a single sample to a category, it transforms it into a similarity problem between pairs of samples. This approach is less dependent on precise labels and offers greater robustness against noise.
Also Read:
- DynaMind: Decoding Dynamic Visual Experiences from Brain Signals with Unprecedented Fidelity
- Deep Cluster Atlas: A New Approach to Personalized Brain Mapping
Inference and Results
During the inference stage, when a new, unseen target subject’s EEG data is presented, the MATL-DC model uses the optimally trained domain and class prototypes. It first determines which superdomain the new sample’s domain features belong to, and then, within that superdomain, it identifies the emotional state using the class prototypes. This entire process occurs without any prior exposure to the target domain data.
The MATL-DC framework was rigorously validated on three widely used public databases: SEED, SEED-IV, and SEED-V. The results demonstrated that MATL-DC achieved comparable or even superior performance compared to existing state-of-the-art methods that actually rely on target domain data during training. For instance, on the SEED dataset, MATL-DC achieved an accuracy of 84.70%, outperforming the best traditional machine learning model by over 13% and the next best deep learning model by 1.42%. Similar improvements were observed across SEED-IV and SEED-V datasets, under both single-session and cross-session validation protocols.
Ablation studies confirmed the effectiveness of each component of MATL-DC, showing significant performance drops when any module (like domain prototypes, discriminators, multi-domain aggregation, or pairwise learning) was removed. Furthermore, experiments with artificially introduced label noise demonstrated the remarkable robustness of the pairwise learning strategy, with the model’s performance decreasing by only 3.32% even with a 30% noise rate, compared to a 6.69% drop for traditional pointwise learning.
This research provides a promising and highly practical solution for EEG-based emotion recognition, especially in scenarios where data from new subjects is unavailable during model training. The source code for MATL-DC is publicly available, fostering further research and application. You can find more details in the full research paper: MATL-DC: A Multi-domain Aggregation Transfer Learning Framework for EEG Emotion Recognition with Domain-Class Prototype under Unseen Targets.
