TLDR: SEADIALOGUES is a new open-source dataset of 32,000 multi-turn dialogues in eight Southeast Asian languages, designed to make AI conversational agents more culturally aware. It incorporates local cultural nuances, personas, and topics, addressing the gap left by existing datasets that often overlook cultural sensitivity. The dataset was created using a pipeline involving LLMs and human annotation, showing that proprietary models currently outperform open-source ones in generating culturally relevant dialogues.
In the rapidly evolving field of artificial intelligence, dialogue systems have made remarkable progress, enabling everything from task-oriented assistants to casual chit-chat bots. However, a significant challenge remains: most existing datasets for these systems often overlook the rich and subtle cultural nuances that are fundamental to natural human conversations. This oversight can lead to AI models that struggle to reflect cultural values accurately or generate contextually appropriate responses, especially in diverse regions.
Addressing this critical gap, a new research initiative introduces SEADIALOGUES, a groundbreaking dataset specifically designed to foster culturally grounded dialogue systems. This dataset focuses on Southeast Asia, a region home to over 700 million people and immense cultural diversity, where many languages are considered low-resource despite having large speaker populations.
What is SEADIALOGUES?
SEADIALOGUES is an open-source, multilingual, multi-turn dialogue dataset featuring conversations in eight languages from six Southeast Asian countries: Indonesian, Javanese, Minangkabau, Thai, Malay, Vietnamese, Tamil, and Tagalog. It comprises 32,000 dialogues, covering more than 100 culturally relevant topics. To enhance cultural relevance and personalization, each dialogue incorporates persona attributes and two culturally grounded topics that mirror everyday life in the respective communities. The dataset also includes 210 diverse personas to support personalized and culturally aware dialogue generation.
How is it Different?
Unlike many existing dialogue datasets that rely on direct translation from English or simple entity replacement, SEADIALOGUES integrates manually curated cultural knowledge directly into its design. This includes local entities, traditional foods, and communication norms, ensuring that the generated dialogues are coherent, culturally authentic, and reflect real-world user behavior. This approach helps bridge the gap between linguistic diversity and cultural representation in conversational AI.
The Data Generation Process
The creation of SEADIALOGUES involved a meticulous four-stage pipeline. First, scenario and persona templates were generated, with culturally specific elements replaced by abstract placeholders. Second, a process called lexicalization filled these placeholders with culturally relevant entities, ensuring contextual alignment. Third, large language models (LLMs) were prompted with these detailed scenarios and personas to synthetically generate multi-turn dialogues. Finally, these generated dialogues underwent a rigorous annotation process, involving both human annotators and automated LLM-as-judge evaluations, to ensure high quality and cultural appropriateness.
Also Read:
- Enhancing AI Content Moderation: A New Benchmark and Human-Centric Strategy
- Understanding Advanced AI Search Agents
Evaluation and Findings
The dataset’s quality was assessed using several criteria, including fluency, engagingness, coherence, naturalness, cultural relevance, profile detection (how well personas are maintained), and correctness (adherence to topics). Both human annotators and automated evaluation methods (G-Eval, M-Prometheus, and R3 reward models) were employed. The findings indicate that proprietary (closed-source) LLMs, such as Gemini 1.5 Flash and GPT-4o mini, generally outperformed open-source models like Llama-3.1-8B Instruct and Aya-8B Expanse in generating high-quality, culturally relevant dialogues. G-Eval, in particular, showed a good correlation with human annotations, though further improvements are needed for LLM judges to fully match human evaluations for Southeast Asian dialogues.
The introduction of SEADIALOGUES marks a significant step forward in developing more culturally aware and human-centric large language models. By providing a rich, open-source resource, it aims to support advancements in conversational AI, especially for underrepresented languages and cultures. For more detailed information, you can refer to the full research paper here.
