TLDR: The VLSP 2025 MLQA-TSR Challenge focused on Vietnamese Multimodal Legal Question Answering on Traffic Sign Regulation. It involved two subtasks: multimodal legal retrieval and multimodal question answering. Researchers developed AI systems to interpret both text and images of traffic signs to answer legal questions. The challenge provided a new benchmark dataset and saw top systems achieve an F2 score of 64.55% for retrieval and an accuracy of 86.30% for question answering, pushing the boundaries of AI in legal text processing for low-resource languages.
The VLSP 2025 MLQA-TSR Challenge, short for Vietnamese Multimodal Legal Question Answering on Traffic Sign Regulation, recently concluded, marking a significant step forward in the field of artificial intelligence for legal text processing, particularly for low-resource languages like Vietnamese. This challenge brought together researchers to develop intelligent systems capable of understanding and integrating both text and images to answer complex legal questions related to traffic signs in Vietnam. You can find the full research paper here.
Addressing a Complex Domain
Multimodal Question Answering (QA) is a demanding task in Natural Language Processing (NLP) that requires AI systems to process and combine different types of data, such as text and images, to find accurate information. Legal text processing adds another layer of difficulty due to its formal language, intricate structure, and specialized terminology. The VLSP 2025 MLQA-TSR challenge specifically focuses on Vietnamese traffic sign regulations, requiring systems to interpret visual traffic signs alongside textual legal questions and consult relevant statutes like the National Technical Regulation on Traffic Signs and Signals and the Law on Road Traffic Order and Safety.
Two Key Subtasks
The challenge was divided into two main subtasks:
- Subtask 1: Multimodal Legal Retrieval – Given a question that includes both text and an image of a traffic sign, the goal was for systems to identify and rank the most relevant articles from a legal database. This task was evaluated using the F2-score, which prioritizes recall to ensure that relevant articles are not missed.
- Subtask 2: Multimodal Legal Question Answering – For a given multimodal question and a list of relevant legal articles, systems had to predict the correct answer from four multiple-choice options. Accuracy was the primary metric for this subtask.
Building the Benchmark Dataset
A comprehensive dataset was meticulously constructed through a multi-stage process. This involved collecting images of Vietnamese traffic signs, followed by a rigorous annotation process where eight undergraduate students created questions and answers based on these images and legal documents. They also identified relevant legal articles. A crucial cross-checking and validation stage ensured the quality and consistency of the annotated data. The final dataset was split into training, public test, and private test sets, and included a law database with articles from two key Vietnamese legal documents, one of which contained both images and tables.
Innovative Approaches from Participants
Participants in the challenge employed a variety of advanced techniques. For multimodal legal retrieval, common strategies included creating contextual multimodal embeddings using models like CLIP, Jina Embedding, and C-RADIOv2, often enhanced by traffic sign detection models such as YOLO or OWL to focus on key visual information. Data preprocessing, including text normalization and HTML table transformation, was also vital. Vector databases like Qdrant and FAISS were frequently used for efficient similarity searching. Some teams even explored graph-based representations to capture semantic relationships within the multimodal data.
For the multimodal legal question answering task, vision-enabled Large Language Models (LLMs) like Qwen2.5-VL, InternVL3, Gemma2, and LLaMa3.2-Vision were widely adopted. Participants utilized various prompting techniques, including zero-shot, few-shot, and Chain-of-Thought prompting, to guide these LLMs in generating accurate answers.
Also Read:
- AI’s Multimodal Safety Challenge: New Research Exposes Gaps in Joint Image-Text Understanding
- A New Standard for Legal AI: Unveiling the Massive Legal Embedding Benchmark
Key Outcomes and Future Directions
The challenge yielded impressive results. In Subtask 1 (Multimodal Legal Retrieval), the top-performing team, LifeIsTough, achieved an F2 score of 64.55%. For Subtask 2 (Multimodal Legal Question Answering), the Berry team secured the highest accuracy at 86.30%. These results demonstrate the efficiency and robustness of the proposed methodologies, with all participating teams outperforming the baseline methods.
Comparisons with other legal NLP competitions, such as ALQAC 2024 and COLIEE 2024, highlighted that legal retrieval remains a more challenging task, especially in a multimodal context, due to the complex structure and specialized terminology of legal documents. The VLSP 2025 MLQA-TSR challenge has successfully provided a valuable benchmark dataset and fostered innovative research in Vietnamese multimodal legal text processing, paving the way for more intelligent systems in the legal domain.
