TLDR: Researchers have developed a RAG-based AI system to help archaeologists determine the origin and age of ancient artifacts. The system uses multimodal retrieval (raw images, edge-enhanced images, and semantic text/image matching) to find similar objects in a vast knowledge base. A vision-language model then synthesizes this information to infer chronological, geographical, and cultural attributions. Expert evaluations show the system is particularly strong at generating accurate historical and geographical conclusions, providing valuable support for archaeological provenance analysis.
Archaeological research often involves the challenging task of determining where and when ancient artifacts originated. This process, known as provenance analysis, traditionally relies heavily on the individual expertise of scholars who compare unknown objects against vast collections of reference materials. However, this method can be time-consuming, subjective, and limited by access to information.
A new study introduces a novel system designed to assist experts in this complex task. Developed by Tuo Zhang, Yuechun Sun, and Ruiliang Liu, this system leverages a combination of multimodal retrieval and advanced vision-language models to systematically analyze archaeological artifacts. The goal is to provide concrete starting points for analysis, reduce the cognitive burden on researchers, and enhance the transparency and objectivity of provenance attribution.
How the System Works
The system operates through three main components:
External Knowledge Construction: The foundation of the system is a comprehensive knowledge base built from archaeological monographs, catalogs, and excavation reports. These documents, often in PDF format, are converted into structured files for efficient text retrieval. Crucially, the system also extracts and indexes figures, plates, and visual layouts, creating an image repository to support image-based searches, especially for low-quality scanned documents where text extraction might be unreliable.
Multimodal Retrieval: To find similar artifacts, the system employs three parallel retrieval strategies:
- Raw Image Retrieval: Directly compares the input artifact image with all reference images in the database to find visually similar objects.
- Edge-Enhanced Retrieval: Applies a Gaussian filter to highlight structural edges and contours in the query image, which is particularly useful for matching line drawings and archaeological sketches.
- Semantic CLIP-based Retrieval: Embeds both text and images into a shared space, allowing the system to find matches based on semantic alignment, even if visual appearances differ but descriptive content is similar.
After retrieval, the results from all three strategies are combined, deduplicated, and filtered to select the most relevant candidates.
Inference and Reasoning: The filtered candidates are then passed to a Vision-Language Model (VLM), specifically GPT-4o, for structured analysis. In the first phase, the VLM interprets each candidate individually, inferring attributes like likely excavation site, estimated cultural period, and bibliographic references. In the second phase, the VLM synthesizes all this information, along with the target image, to determine the most probable excavation site and historical period for the object. It also identifies the most relevant supporting reference and provides a detailed justification for its predictions, highlighting key visual and textual cues.
Also Read:
- Unlocking Geometric Understanding in AI with GeoRef
- Enhancing Language Models with Specialized Knowledge through Iterative Reinforcement Learning
Evaluation and Impact
The system was evaluated on a collection of Eastern Eurasian Bronze Age artifacts from the British Museum, using reference materials in multiple languages. Expert evaluators assessed the system’s performance on identifying stylistically similar objects and generating chronological, geographical, and cultural attributions.
While visual similarity identification showed mixed results, the system demonstrated significantly stronger performance in generating chronological and geographical conclusions. Nearly 46% of the AI-generated attributions achieved a high score (Score 3 or higher) from experts. This indicates that the system excels at synthesizing typological information into scholarly conclusions, making it a valuable tool for supporting expert decisions rather than replacing traditional comparative analysis.
This innovative RAG-based system offers archaeologists and art historians a powerful new tool to navigate vast comparative corpora, providing interpretable and actionable outputs that can significantly streamline the provenance analysis process. For more details, you can read the full research paper here.
