TLDR: A new framework called MEEtBrain uses AI-generated music and a portable EEG-fNIRS headband to objectively assess music-induced emotions. It addresses limitations of previous studies by providing diverse music stimuli and collecting multimodal brain signals with a user-friendly, wearable device. The research validated that AI-generated music effectively evokes target emotions and demonstrated improved emotion recognition accuracy by combining EEG and fNIRS data.
Emotions play a crucial role in our mental well-being, and music has long been recognized as a powerful tool for influencing our feelings. However, objectively understanding how music affects our emotions, especially through brain signals, has faced several challenges. Traditional studies often rely on limited music selections, use bulky equipment, or only capture one type of brain signal, which restricts their real-world application.
A new research paper introduces a groundbreaking framework called MEEtBrain, designed to overcome these limitations. This innovative system integrates AI-generated music with a portable, wireless headband that simultaneously collects electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals. The goal is to provide a more accessible and comprehensive way to assess emotions induced by music.
Addressing Key Limitations
The MEEtBrain framework tackles three main issues in current music-based emotion research:
First, **Stimulus Constraints**: Instead of relying on small, pre-selected music libraries that might have copyright issues or subjective biases, MEEtBrain uses AI-Generated Content (AIGC) to create music on a large scale. Researchers designed prompt templates based on the Valence-Arousal model (which categorizes emotions by pleasantness and intensity) to automatically generate diverse music clips. This ensures a wide range of stimuli without selection biases.
Second, **Modality Specificity**: Many studies use only one type of neural data, like EEG, which excels in temporal resolution but lacks spatial specificity. MEEtBrain addresses this by fusing EEG and fNIRS signals. fNIRS provides complementary insights by measuring changes in blood oxygenation in the brain, offering a more complete picture of brain activity related to emotions.
Third, **Portability Limitation**: Traditional setups often involve cumbersome equipment, like gel-based EEG caps with many channels, making them impractical for everyday use. MEEtBrain introduces a lightweight, headband-style device with dry electrodes, making it easy to wear and user-friendly. This enhances the framework’s real-world applicability and accessibility for emotion regulation.
How MEEtBrain Works
The process begins with creating a vast library of AI-generated music. Using a model like MUSICGEN, prompts describing specific emotions, instrument styles, and contextual scenarios are fed in to produce hundreds of unique music clips. These clips are then screened by volunteers to ensure they effectively evoke the intended emotions, resulting in a curated set of 101 music clips categorized into four emotion types: High Arousal High Valence (HAHV), High Arousal Low Valence (HALV), Low Arousal High Valence (LAHV), and Low Arousal Low Valence (LALV).
Next, participants listen to these music clips while wearing the portable EEG-fNIRS headband. This device simultaneously records brain signals from the forehead area. After each music clip, participants provide self-reported ratings of Valence, Arousal, and Liking. This dual approach of objective brain signal measurement and subjective self-reporting helps validate the emotional responses.
Validating the Approach and Recognizing Emotions
The researchers validated their AI-generated music by analyzing its structural features (like tempo, rhythmic articulation, and mode) and found that these features significantly correlated with the intended emotional dimensions, consistent with prior research. Participant ratings further confirmed that the AI-generated music effectively evoked the corresponding emotional states, with most ratings aligning well with the music’s initial emotional labels.
When analyzing the brain signals, the study found significant differences in EEG beta band power across different emotion groups, particularly distinguishing the LAHV group from others. For fNIRS, dynamic changes in hemoglobin concentration (variance metrics) showed significant correlations with emotion labels, suggesting that these changes encode emotion-related information.
For emotion recognition, the study used a deep neural network to classify emotions (binary classification of valence and arousal) based on the collected brain signals. The results showed that combining all modalities (EEG, fNIRS-derived hemodynamic signals, and photoplethysmography or PPG) significantly improved classification performance compared to using single modalities. Interestingly, arousal classification consistently performed better than valence, suggesting that arousal is more reliably detectable using these physiological signals.
Also Read:
- Enhanced Emotion Recognition Through Advanced EEG Feature Selection
- FDC-Net: A Collaborative Approach to EEG Emotion Recognition in Noisy Environments
Future Impact
The MEEtBrain framework represents a significant step forward in affective computing. By addressing the limitations of stimulus diversity, signal modality, and device portability, it paves the way for more practical and accessible emotion regulation technologies. The researchers have also made their multimodal dataset, which includes data from 44 participants, publicly available to encourage further research and real-world applications. You can find more details about this research in the full paper available at this link.
