TLDR: MMBERT is a new AI model designed to detect Chinese hate speech, especially when it’s hidden using cloaking techniques. It combines text, speech, and visual information using a ‘Mixture-of-Experts’ architecture and a unique three-stage training process. This allows it to understand subtle cues that text-only models miss, leading to significantly better and more robust detection performance across various Chinese hate speech datasets.
Online communities face a persistent threat from hate speech, a challenge exacerbated by the anonymity and vast scale of digital platforms. While significant progress has been made in automated hate speech detection, most efforts have focused on English, leaving other major languages like Chinese relatively underserved. Chinese social media platforms, in particular, see widespread use of ‘cloaking techniques’ – subtle manipulations that exploit the structural and phonological properties of the Chinese language to evade detection by conventional text-based systems.
These cloaking perturbations can take various forms, such as ‘Deformation’ (altering character components to change meaning), ‘Homophonic Substitution’ (replacing words with similar-sounding ones), ‘Abbreviation’ (contracting sensitive terms), and ‘Code-Mixing’ (incorporating non-Chinese elements like Pinyin or emojis). These methods make it incredibly difficult for text-only models to accurately identify hateful expressions.
To address this critical gap, researchers have introduced MMBERT, a novel BERT-based multimodal framework. MMBERT is designed for robust Chinese hate speech detection, specifically targeting these challenging cloaking perturbations. It integrates textual, speech, and visual modalities through a sophisticated Mixture-of-Experts (MoE) architecture.
How MMBERT Works
MMBERT’s core innovation lies in its ability to process and understand information from multiple sources simultaneously. Unlike traditional models that might struggle with the nuances of cloaked language, MMBERT leverages a combination of inputs:
- Textual Modality: The primary input, processed by a BERT-based encoder.
- Speech Modality: Audio inputs are processed to capture phonetic cues, which are crucial for detecting homophonic substitutions.
- Visual Modality: Image inputs, such as rendered word-level font images, help in identifying visual alterations or deformations in characters.
The Mixture-of-Experts (MoE) architecture is central to MMBERT. It incorporates modality-specific experts – essentially specialized neural networks for text, speech, and vision. A ‘router’ mechanism dynamically assigns input representations to the most appropriate expert based on the context, allowing the model to adapt its processing based on the type of perturbation present.
A Progressive Training Strategy
Directly integrating MoE into BERT-based models can lead to instability. To overcome this, MMBERT employs a progressive three-stage training paradigm:
- Stage 1: Aligner Training: This initial stage focuses on teaching the model’s ‘aligners’ to map visual and auditory inputs into a shared linguistic space that BERT can understand. This ensures that information from different modalities can be effectively combined.
- Stage 2: Expert Training: In this stage, the modality-specific experts (for text, speech, and vision) are trained independently to specialize in their respective domains, refining their ability to process their unique data types.
- Stage 3: MMBERT Tuning: The final stage integrates all the trained experts into the full MoE-augmented architecture. The entire model, including the context-aware routing mechanism, is fine-tuned on real multimodal Chinese hate speech data, ensuring optimal performance and robustness.
Also Read:
- Deepfake Detection Breakthrough: Unlocking Multi-Modal AI’s Middle Layers for Universal Forensics
- Unveiling the Black Box: A New Framework for Explaining How AI Models Combine Images and Text
Impressive Results and Robustness
Empirical results across several benchmark Chinese hate speech datasets demonstrate MMBERT’s superior performance. It significantly surpasses fine-tuned BERT-based encoder models, fine-tuned Large Language Models (LLMs), and LLMs utilizing in-context learning approaches. Notably, MMBERT achieves state-of-the-art results, particularly in detecting cloaked adversarial content, highlighting the value of its multimodal modeling and progressive training approach.
Ablation studies further confirm the importance of each component. The progressive training strategy is crucial for stable convergence and strong generalization. Furthermore, the studies revealed that speech features contribute more significantly than visual features in capturing adversarial cues introduced by cloaking perturbations, underscoring the critical role of speech in improving robustness for Chinese hate speech detection.
MMBERT represents a significant step forward in addressing the complex challenge of Chinese hate speech detection, especially against sophisticated cloaking techniques. By integrating multiple modalities and employing a carefully designed training strategy, it offers a more robust and effective solution for content moderation in Chinese online communities. For more details, you can refer to the full research paper: MMBERT: Scaled Mixture-of-Experts Multimodal BERT for Robust Chinese Hate Speech Detection under Cloaking Perturbations.
