TLDR: KOTOX is a new Korean dataset designed to help language models better detect and neutralize toxic content that has been intentionally obscured. It categorizes Korean-specific obfuscation techniques into 17 rules across five linguistic categories, creating easy, normal, and hard versions of toxic-neutral text pairs. This dataset enables models to learn deobfuscation and detoxification simultaneously, addressing a critical gap for low-resource languages like Korean where existing models struggle with hidden toxicity.
Online communication has brought with it a significant challenge: the spread of toxic content. While many efforts have been made to detect and neutralize such content, most research has focused on English, leaving other languages, especially those with fewer resources, underserved. This gap means that large language models (LLMs) often struggle to identify and clean up toxic expressions in these languages, a problem made worse when users employ clever obfuscation techniques to hide their harmful messages.
To tackle this issue, researchers from Yonsei University have introduced KOTOX, a groundbreaking Korean Toxic Dataset designed for both deobfuscation and detoxification. This dataset is unique because it simultaneously supports both tasks for the Korean language, which is an agglutinative language with a highly transformable phonemic and compositional script called Hangeul, making it particularly suitable for diverse obfuscation patterns.
The KOTOX dataset categorizes various obfuscation approaches based on the unique linguistic characteristics of Korean. The researchers defined a set of 17 transformation rules grounded in real-world examples. These rules are grouped into five main categories: Phonological, Iconological, Transliteration-based, Syntactic, and Pragmatic. For instance, phonological rules exploit the similarity in pronunciation, while iconological rules leverage visual similarities in Hangeul characters. Transliteration-based rules convert text into other languages that share pronunciation or meaning, and syntactic rules manipulate spacing or syllable order. Pragmatic rules involve inserting irrelevant elements like symbols or emojis to alter perceived sentiment.
Using these rules, the team constructed three versions of the dataset—easy, normal, and hard—each representing different levels of obfuscation difficulty. Starting with 2.3K neutral-toxic source pairs from an existing dataset called K/DA, they applied these rules to create approximately 6.9K high-quality Korean neutral-toxic pairs, each with a corresponding obfuscated counterpart. This systematic approach ensures a comprehensive resource for studying obfuscated toxic content.
KOTOX serves as a unified resource for three complementary tasks: Obfuscated Toxic Text Classification, Neutral Text Deobfuscation, and Obfuscated Toxic Text Sanitization. These tasks are designed to jointly model toxicity and obfuscation, providing a robust framework for evaluating how well language models can handle hidden toxic content. The dataset’s paired and level-based design allows for aligned supervision across these tasks and enables controlled analysis of model robustness.
Experiments conducted with KOTOX revealed that existing LLMs, typically trained on clean text, have limited understanding of obfuscated Korean text. Models fine-tuned on KOTOX, however, showed significant improvements in detecting obfuscated toxic text, deobfuscating neutral text, and sanitizing toxic content. This demonstrates the dataset’s effectiveness in enhancing model robustness and comprehension.
Also Read:
- Adaptive AI Framework Enhances Detection of Subtle Hate Speech
- LatentBreak: A Stealthy New Method for Bypassing LLM Safety Measures
The researchers emphasize that KOTOX is the first dataset of its kind to simultaneously support deobfuscation and detoxification for Korean. They anticipate that it will significantly contribute to a deeper understanding and mitigation of obfuscated toxic content in LLMs for low-resource languages. The code and data for KOTOX are publicly available on GitHub. You can read the full research paper for more details. Read the full paper here.
