TLDR: GRAFT is a new document-level machine translation framework that uses Large Language Model (LLM) agents and a graph-based approach to improve translation quality. It segments documents into discourse units, models dependencies between them using a directed acyclic graph (DAG), and employs memory agents to ensure consistency and coherence. Experiments show GRAFT significantly outperforms state-of-the-art systems across various languages and domains by effectively handling complex discourse phenomena.
Document-level Machine Translation (DocMT) aims to translate entire documents while maintaining consistency, coherence, and the overall meaning across sentences and paragraphs. While significant progress has been made in translating individual sentences, handling longer texts remains a challenge due to complex linguistic phenomena that span multiple sentences, such as pronoun resolution, consistent terminology, and logical flow.
Traditional DocMT methods often struggle because they rely on simple rules to break down documents into smaller units, which don’t always capture the true underlying structure needed for accurate translation. Other systems might fail to keep the translation consistent throughout the document.
Introducing GRAFT: A New Approach to Document Translation
To overcome these hurdles, researchers have introduced the Graph-Augmented Agentic Framework for Document-Level Translation, or GRAFT. This innovative system uses Large Language Model (LLM) agents and a unique graph-based approach to translate documents more effectively. GRAFT combines three key elements: intelligent document segmentation, modeling dependencies between parts of the document using a directed acyclic graph (DAG), and discourse-aware translation.
GRAFT’s design involves four specialized LLM agents working together:
- Discourse Agent: This agent intelligently segments the document into self-contained “discourse units” using an LLM-based algorithm. Each unit becomes a node in the DAG structure. This is crucial because it ensures each segment is coherent on its own, addressing issues like word sense disambiguation and idioms within that unit.
- Edge Agent: After segmentation, this agent identifies and establishes directed connections (edges) between discourse units. These connections represent instances where information needs to be shared between units for accurate translation, capturing inter-discourse phenomena like pronoun antecedents and logical relations.
- Memory Agent: For each translated discourse unit, this agent extracts and structures “local memory.” This memory includes mappings of nouns to pronouns, entities from source to target language, phrase-to-phrase mappings, and translations of discourse connectives. It also provides a concise summary of the discourse context.
- Translation Agent: This is the core translator. It takes a discourse unit and combines it with relevant context from its “incident memories” (information from preceding, related discourse units, prioritizing earlier dependencies). It then uses an LLM to produce a contextually informed translation. After translation, the Memory Agent updates its local memory, ensuring subsequent translations benefit from the latest contextual information.
This multi-agent, graph-augmented pipeline allows GRAFT to explicitly model and propagate context throughout the document, leading to more coherent and consistent translations. The system transforms a source document into a DAG of discourses, then iteratively processes and translates these discourses, building a rich memory of translated context as it goes.
Also Read:
- Assessing LLM Agent Memory: A New Benchmark for Interactive Intelligence
- CodeAgents: Boosting LLM Agent Performance and Efficiency with Codified Reasoning
Performance and Impact
Experiments conducted across eight translation directions (e.g., English to Chinese, German to English) and six diverse domains (Fiction, News, Social, TED, Novel) show that GRAFT significantly outperforms existing document-level machine translation systems. For instance, GRAFT achieved an average improvement of 2.8 d-BLEU on TED test sets and 2.3 d-BLEU for domain-specific translation from English to Chinese, compared to strong baselines.
The research highlights GRAFT’s consistent ability to handle complex discourse-level phenomena, resulting in translations that are not only accurate but also coherent and contextually appropriate. Even with smaller LLM backbones, GRAFT demonstrated competitive performance compared to larger, closed-source models like OpenAI GPT models, showcasing the efficiency of its unique segmentation and dependency strategies.
GRAFT also excels in domain-specific translation, which often involves specialized terminology and complex co-reference patterns. In Chinese-to-English translation across News, Social, Fiction, and Q&A domains, GRAFT consistently outperformed baselines, showing superior handling of terminology consistency and pronoun resolution accuracy. For example, it achieved an average gain of 6.0% in terminology consistency and 6.0% in zero pronoun translation accuracy over GPT-4o-mini.
The study also confirmed the importance of each of GRAFT’s components through ablation studies, where removing any part led to a noticeable drop in performance. This underscores the integrated nature and effectiveness of the Discourse, Edge, and Memory Agents in the overall pipeline.
While GRAFT’s multi-agent design introduces some computational overhead, leading to higher latency and cost compared to monolithic systems, its significant improvements in translation quality make it a compelling choice for applications where high precision and contextual accuracy are paramount. Future work aims to optimize its architecture for broader use cases.
For more in-depth details, you can read the full research paper available at arXiv:2507.03311.
