TLDR: EMOD is a novel framework for EEG-based emotion recognition that addresses the challenges of diverse datasets by unifying different emotion labels into a continuous Valence-Arousal space and employing a flexible neural network architecture to handle various EEG data formats. Through V-A guided contrastive learning and a Triple-Domain Encoder with a Spatial-Temporal Transformer, EMOD learns transferable and emotion-aware representations. Pretrained on 8 public datasets and evaluated on 3 benchmarks, EMOD achieves state-of-the-art performance with fewer parameters, demonstrating strong generalization and efficiency in diverse emotion recognition scenarios.
Emotion recognition from brain signals, specifically electroencephalography (EEG), is a vital area in affective computing. This field aims to bridge the gap between human emotions and intelligent systems, enabling more natural and adaptive human-computer interactions. While deep learning has shown promise in this area, existing models often struggle with generalization across different EEG datasets. This is primarily due to two major challenges: the varied ways emotions are labeled (label heterogeneity) and the different ways EEG data is collected and formatted (data format heterogeneity).
Traditional approaches typically require models to be specifically designed and trained for each dataset. This means that a model trained on one dataset might not perform well on another, limiting its practical application when new EEG sources emerge. Emotion labels can be discrete (like ‘happy’, ‘sad’, ‘fear’) or continuous (Valence-Arousal scores), and even within the same type, definitions and value ranges can vary. Similarly, EEG acquisition protocols differ widely, with varying numbers of channels and sampling rates, forcing models to adopt dataset-specific architectures.
To address these significant challenges, researchers have proposed EMOD: A Unified EEG Emotion Representation Framework Leveraging Valence-Arousal (V-A) Guided Contrastive Learning. EMOD is designed to learn transferable and emotion-aware representations from diverse EEG datasets by tackling both semantic and structural inconsistencies. You can find the full paper here: EMOD Research Paper.
How EMOD Unifies Emotion Labels
EMOD tackles label heterogeneity by projecting all emotion annotations, whether discrete or continuous, into a unified Valence-Arousal (V-A) space. The V-A model is a widely accepted framework that describes emotions along two continuous dimensions: Valence (how pleasant or unpleasant an emotion is) and Arousal (how intense or calm an emotion is). For datasets with discrete labels, EMOD estimates their V-A values, effectively creating a shared emotional reference point for all data. Based on this unified space, EMOD uses a unique soft-weighted supervised contrastive loss. This loss function encourages EEG samples that are emotionally similar in the V-A space to cluster together in the model’s learned representation, while pushing dissimilar samples apart. Unlike traditional methods that treat all samples within a class as equally positive, EMOD assigns soft similarity weights based on V-A proximity, allowing for a more nuanced understanding of emotional relationships.
A Flexible Architecture for Diverse EEG Data
To handle the variations in EEG data formats, EMOD employs a flexible and unified architecture. This backbone consists of a Triple-Domain Encoder followed by a Spatial-Temporal Transformer. The Triple-Domain Encoder is designed to extract emotion-relevant features from three complementary domains: temporal (how signals change over time), spectral (the frequency components of the signals), and spatial (where the activity occurs on the scalp). This ensures that EMOD can robustly process EEG signals regardless of their original channel configurations or sampling rates.
The Spatial-Temporal Transformer then models long-range dependencies across both spatial and temporal dimensions. It uses an axial attention mechanism, which separately captures relationships between EEG channels (spatial) and dynamics over time (temporal). A key innovation here is the use of relative temporal encoding, which allows the model to understand temporal dependencies based on relative positions rather than absolute time, making it more adaptable to varying segment lengths and signal characteristics.
Pretraining and Performance
EMOD is pretrained on eight publicly available EEG-based emotion datasets, encompassing a wide range of acquisition protocols and annotation schemes. This extensive pretraining allows the model to learn generalizable features before being fine-tuned for specific emotion recognition tasks. The framework was evaluated on three benchmark datasets: FACED, SEED-V, and SEED.
Experimental results demonstrate that EMOD consistently achieves state-of-the-art performance across all evaluation datasets. For instance, on the FACED dataset, EMOD significantly outperformed the strongest baseline by a considerable margin in balanced accuracy, Cohen’s Kappa, and weighted F1-score. What’s more, EMOD is remarkably lightweight, using significantly fewer parameters than many recent EEG foundation models, and requires less pretraining data. This suggests that emotionally guided contrastive learning on specific emotion datasets is highly effective for learning robust emotion representations.
Ablation studies further confirmed the importance of EMOD’s core components. Removing the V-A guided soft-weighted contrastive loss or any part of the flexible architectural design led to a noticeable drop in performance, highlighting their crucial roles in the framework’s success. Visualizations of the learned features also showed that EMOD’s pretraining effectively organizes EEG samples into structured, well-separated clusters that align with the continuous Valence-Arousal emotional gradient.
Also Read:
- NeuroBridge Unlocks New Potential in Decoding Visuals from Brain Signals
- Optimizing Sleep Analysis for Wearable Devices
Conclusion
EMOD represents a significant step forward in EEG-based emotion recognition. By unifying heterogeneous emotion labels through Valence-Arousal guidance and employing a flexible architecture to handle diverse EEG data formats, EMOD offers a robust, generalizable, and efficient solution. This work underscores the potential of emotionally guided pretraining to learn highly sensitive and transferable EEG representations for affective computing applications.
