TLDR: A new research paper introduces a method for detecting and extracting Latin fragments from 18th-century historical documents using Large Language Models (LLMs) and Multimodal LLMs (MLLMs). The study benchmarks these models against a novel dataset of 724 annotated pages, demonstrating that reliable Latin detection is achievable. While open-source LLMs show strong performance, the models primarily rely on statistical patterns rather than deep semantic understanding, leading to challenges with short fragments and certain categories of Latin usage. The work provides a practical pipeline for large-scale historical research and highlights the potential of AI in digital humanities.
A new study delves into the complex task of automatically identifying Latin text within historical documents, a crucial step for researchers studying the evolution of languages and ideas during the early modern period. The paper, titled “Detecting Latin in Historical Books with Large Language Models: A Multimodal Benchmark,” introduces a novel approach using advanced artificial intelligence models to tackle this challenging problem.
Latin held a dominant position as the written language in Western Europe for over a thousand years, gradually giving way to local languages. However, even as vernaculars gained prominence, Latin fragments frequently appeared in texts—as direct quotes, specialized terms, or instances of code-switching. Extracting these Latin snippets from vast historical archives is vital for understanding linguistic shifts and the interplay between classical and modern thought.
The researchers focused on documents from the Eighteenth Century Collection Online (ECCO) corpus, which presents significant challenges due to varied page layouts, inconsistent print quality, and noisy text extracted by Optical Character Recognition (OCR). To address the lack of suitable data, the team created a unique dataset of 724 manually annotated pages, validated by 18th-century publishing culture specialists. This dataset categorizes Latin usage into 12 distinct types, ranging from direct quotes and independent Latin texts to footnotes, code-switching, and legal or ecclesiastical formulae.
The study explores the capabilities of modern Large Language Models (LLMs), including multimodal models (MLLMs) that can process both text and images. These models are evaluated on two main tasks: first, detecting whether a page contains any Latin (page-level detection), and second, extracting the specific Latin text segments. The models were tested using a unified, prompt-based pipeline with different input modalities: text-only (OCR text), image-only (scanned page image), and multimodal (both image and text).
The findings are quite remarkable. Several large open-source LLMs, such as DeepSeek-R1 and Qwen3, not only outperformed a traditional statistical language identifier (Lingua) but also surpassed proprietary models like GPT-4.1. This highlights the rapid advancements in open-source AI development. The research indicates that reliable Latin detection in challenging historical materials is indeed achievable with contemporary models. Multimodal inputs, combining both image and text, generally improved performance, demonstrating the benefit of visual cues in noisy, OCR-heavy documents.
However, the study also revealed limitations. Models performed exceptionally well on longer, more distinct Latin text types, but struggled with shorter fragments found in categories like code-switching, dictionaries, or side-notes. This suggests that the models primarily rely on statistical patterns of vocabulary and text rather than a deep functional understanding of the Latin text’s purpose. For instance, models often misidentified Roman names or common English loanwords derived from Latin (like “e.g.” or “etc.”) as Latin, indicating a definitional mismatch with the annotation guidelines.
Also Read:
- Unlocking Folktale Secrets: How AI is Automating Cinderella Motif Analysis
- Mapping Scientific Trends: An LLM Approach to Engineering Research in PNAS
Despite these challenges, the number of erroneously extracted Latin tokens on non-Latin pages was generally small, suggesting that simple post-processing could mitigate over-detection issues. The high performance of image-only models for page-level detection also opens the door for efficient processing of vast historical archives that lack OCR text. This research establishes a strong foundation for future work, including applying the pipeline to the entire ECCO collection and extending the methodology to other historical corpora. For more details, you can read the full research paper here.
