TLDR: This research introduces the first systematic study of text-only End-of-Turn (EOT) detection for real-time Thai voice agents. It compares zero-shot/few-shot LLMs with fine-tuned transformers, demonstrating that small, fine-tuned models, especially Thai-specific ones like Llama3.2-Typhoon2-1B, achieve high accuracy and low latency (e.g., 110ms) by analyzing linguistic cues. This approach significantly improves the fluidity and naturalness of human-computer interactions in Thai by moving beyond traditional silence-based detection.
In the world of voice agents and conversational AI, a smooth and natural interaction is key. One of the biggest challenges has been accurately determining when a user has finished speaking, known as End-of-Turn (EOT) detection. Traditional methods often rely on detecting silence, which can introduce delays and struggle with natural speech patterns like hesitations or language-specific nuances. Imagine a voice assistant that cuts you off mid-sentence or waits too long to respond – it’s frustrating and unnatural.
A recent research paper, “Thai Semantic End-of-Turn Detection for Real-Time Voice Agents,” addresses this very issue, focusing specifically on the Thai language. This work, conducted by Thanapol Popit, Natthapath Rungseesiripak, Monthol Charattrakool, and Saksorn Ruangtanusak, presents the first systematic study of text-only EOT detection for real-time agents in Thai. Their goal is to reduce latency and make human-computer interactions more fluid by analyzing the linguistic content of an utterance rather than just acoustic cues.
The core idea behind semantic EOT detection is to understand if a sentence or thought is complete based on the words themselves. This is particularly important for languages like Thai, which have unique linguistic cues such as sentence-final particles that signal completion. The researchers formulated EOT as a binary decision at token boundaries, meaning the system decides “end” or “not-end” as it processes the transcribed text.
Approaches Explored
The study compared several approaches to EOT detection, including zero-shot and few-shot prompting of compact Large Language Models (LLMs) against supervised fine-tuning of lightweight transformers. They used transcribed subtitles from the YODAS corpus, a public dataset, and applied specific preprocessing steps to clean the data and align it with potential turn boundaries. This involved filtering out noise like songs and advertisements and segmenting subtitle lines into sentence-like units using an instruction-tuned LLM.
Two main paradigms were evaluated: decoder-only models and encoder-only models. Decoder models, often used in generative tasks, were tested with zero-shot/few-shot prompting (where the model uses its existing knowledge to classify without specific training examples, or with a few examples) and also through fine-tuning. Encoder models, which excel in classification tasks, were fine-tuned for explicit binary classification.
Key Findings
The research revealed a clear accuracy-latency trade-off. Supervised fine-tuning consistently yielded the highest performance. For instance, a fine-tuned Llama3.2-Typhoon2-1B model achieved an F1-score of 0.881, demonstrating that task-specific training is crucial for state-of-the-art EOT detection. Compact models like the fine-tuned Qwen3-0.6B also performed very well, offering comparable accuracy at even faster speeds, making them suitable for on-device applications.
Instruction-based prompting (zero-shot and few-shot) was found to be impractical for real-time EOT due to prohibitively high inference latency (1.5–2.6 seconds). This means that while LLMs have an intrinsic understanding of language, they are too slow for instant turn detection without specialized training.
An interesting insight from the study was the importance of threshold calibration for decoder models. Out-of-the-box, these models performed poorly with a default threshold. However, after calibrating the decision threshold on a validation set, their performance became respectable, highlighting that zero-shot thresholding requires a data-driven calibration step.
The comparison between Thai-specific “specialist” models and multilingual “generalist” models also provided nuanced results. In zero-shot scenarios, Thai-specialist models like Typhoon clearly outperformed generalist models, suggesting that language-specific pretraining is vital when fine-tuning isn’t an option. However, with sufficient fine-tuning data, powerful generalist architectures like mDeBERTa-v3-base could surpass specialist models, indicating that a modern, capable model architecture can be more important than language-specific pretraining if fine-tuning is applied.
Also Read:
- Enhancing Conversational AI with KAME’s Tandem System
- Flamed-TTS: Advancing Zero-Shot Text-to-Speech with Efficiency and Naturalness
Implications for Real-Time Voice Agents
This research provides actionable insights for developing real-time Thai EOT detection systems. The fine-tuned Llama3.2-Typhoon2-1B model emerged as an optimal choice, offering high accuracy with low CPU latency (110ms), making it ideal for real-time voice agents. For highly constrained devices, the fine-tuned Qwen3-0.6B is a strong alternative with even faster latency (90ms).
The work demonstrates that lightweight, fine-tuned transformers can provide a fast and accurate semantic EOT signal, moving beyond the limitations of silence-based endpointing. By integrating such models, conversational AI systems can achieve more natural and fluid turn-taking, significantly enhancing the user experience. For more technical details, you can read the full paper here.
