TLDR: This research explores using Large Language Models (LLMs) for Named Entity Recognition (NER) in historical texts, which traditionally lack annotated data. The study finds that LLMs, particularly with few-shot prompting (even a single example), significantly improve performance over zero-shot methods. While LLMs don’t yet match fully supervised models, they offer a cost-effective, language-agnostic solution for information extraction in low-resource historical corpora.
Named Entity Recognition (NER) is a fundamental task in natural language processing (NLP) that involves identifying and classifying specific entities in text, such as people, organizations, locations, and dates. This process is crucial for extracting valuable information from unstructured data, which then supports various applications like information retrieval and question answering.
Traditionally, NER systems rely on supervised machine learning, demanding extensive amounts of annotated training data. However, historical texts present a unique set of challenges. These documents often suffer from inconsistent spelling, archaic vocabulary, and noise introduced during digitization (like optical character recognition errors). More critically, annotated datasets for historical texts are scarce or non-existent due to the high cost and specialized expertise required for manual labeling.
A recent study, “NAMED ENTITY RECOGNITION OF HISTORICAL TEXT VIA LARGE LANGUAGE MODEL” by Shibingfeng Zhang and Giovanni Colavizza, investigates the potential of Large Language Models (LLMs) to overcome these limitations. LLMs, known for their versatility and ability to learn from minimal examples, offer a promising alternative for NER in historical documents, especially in low-resource settings where traditional methods are impractical. You can read the full research paper here.
Exploring LLMs for Historical NER
The researchers explored applying LLMs to NER in historical documents using zero-shot and few-shot prompting strategies. Zero-shot learning requires no task-specific training data, while few-shot learning involves providing the LLM with a small number of annotated examples within the prompt itself. The study utilized the HIPE-2022 dataset, a collection of historical documents spanning the 18th to 20th centuries across five languages (German, French, English, Swedish, and Finnish), manually annotated with entity information.
The methodology involved several steps: first, retrieving similar texts to serve as in-context examples for few-shot learning (this step is skipped for zero-shot). Second, constructing a prompt with these examples and the target text for the LLM (DeepSeek-V3-0324 was used). Finally, processing the LLM’s response into a standard annotation format for evaluation. To ensure robustness, experiments were repeated multiple times, and majority voting was applied to aggregate predictions.
Also Read:
- Annif System Leads GermEval-2025 LLMs4Subjects Task with Hybrid AI for Subject Indexing
- A Collaborative AI Approach to Multimodal Entity Linking
Key Findings and Implications
The experimental results revealed several important insights:
First, few-shot prompting consistently outperformed the zero-shot baseline across all datasets. Even providing a single example significantly improved the LLM’s performance, demonstrating the power of in-context learning.
Second, counterintuitively, increasing the number of examples beyond one (to three or five) often led to a decrease in performance. This phenomenon, sometimes referred to as “lost in the middle,” suggests that longer prompts might exceed the model’s optimal context window or dilute the task’s clarity.
Third, the method of selecting examples (randomly, based on lexical overlap, or embedding similarity) had less impact on performance than the mere presence of an in-context example. This indicates that LLMs can generalize effectively from minimal demonstrations regardless of how those examples are chosen.
Finally, majority voting across multiple runs generally led to modest performance improvements, particularly under fuzzy evaluation settings, which are more lenient about exact entity boundaries.
While LLM-based prompting methods generally did not match the performance of fully supervised state-of-the-art systems (which benefit from extensive training on annotated data), they offer significant practical advantages. These approaches are cost-effective, language-agnostic, and require no fine-tuning or additional training on task-specific data. This makes them particularly valuable for historical NER in low-resource or multilingual scenarios where creating large annotated datasets is infeasible.
In conclusion, this study highlights the promising role of LLMs as a flexible and efficient tool for information extraction from historical texts. Although there is still a performance gap compared to highly specialized supervised models, LLM prompting provides a viable baseline and a powerful alternative for researchers in digital humanities, encouraging further exploration into combining minimal supervision with LLM capabilities for historical text analysis.
