TLDR: A new method uses Large Multi-modal Models (LMMs) to georeference historical biological sample locations by combining text descriptions with gridded maps. This approach significantly outperforms existing methods, achieving an average error of about 1 km, by enabling the AI to visually understand spatial relationships on maps.
Millions of historical biological samples, collected over centuries and stored in natural history collections, often lack precise geographic coordinates. This makes it incredibly difficult for researchers to use this valuable data for biodiversity studies or understanding species distribution over time. Manually adding these coordinates, a process called georeferencing, is a massive and time-consuming task.
Current automated methods for georeferencing text descriptions fall short because they don’t use maps. Humans, when georeferencing, heavily rely on maps to understand spatial relationships like “north of the river” or “near the mountain.” This research paper, titled “Large Multi-modal Model Cartographic Map Comprehension for Textual Locality Georeferencing” by Kalana Wijegunarathna, Kristin Stock, and Christopher B. Jones, introduces a groundbreaking approach that brings maps into the automated georeferencing process. You can read the full paper here: Research Paper.
A New Way to Georeference with AI
The core idea is to use Large Multi-Modal Models (LMMs), which are advanced AI models capable of understanding both text and images. The researchers designed a system where an LMM is given a textual description of a location along with a corresponding map. This allows the AI to “see” the spatial context of the words it’s reading, much like a human would.
How the System Works
The process involves several key steps:
First, the system analyzes the text description to identify place names (like “Lake Wairarapa”) and any spatial relationships (such as “about 400m from lake”).
Next, these identified places are looked up in large geographic databases to get their exact locations and shapes (whether they are a point, a line like a river, or an area like a reserve).
Then, a specific map excerpt is generated, focusing on the area relevant to the description. This map is then divided into a grid, and each grid cell is labeled. The size of these grid cells is also noted, which is important for calculating distances.
Finally, the LMM takes all this information – the original text description, the gridded map, and the grid cell size – and predicts which grid cell is the most likely location described. The researchers found that guiding the LMM with a “step-by-step” thinking process, where it considers the locations and uses the grid size to calculate distances, yielded the best results.
Impressive Results
The new method was tested on a dataset of 25 complex locality descriptions from New Zealand, specifically focusing on those with detailed spatial relations. The results were highly encouraging. The LMM-based approach significantly outperformed existing georeferencing tools and other text-only AI models.
The average distance error for the LMM was approximately 1 kilometer, which is ten times more accurate than the best traditional methods. This means the AI could pinpoint locations much more precisely. A remarkable 60% of the LMM’s predictions were within 1 kilometer of the actual collection site, and 32% landed in the exact correct grid cell. This level of accuracy is crucial for scientific research.
Also Read:
- Boosting Speech AI Performance Through Smart Data Generation
- elsciRL: A New Open-Source Library for Language Integration in Reinforcement Learning
Looking Ahead: Smarter Maps for AI
The study also revealed some fascinating insights and areas for future improvement. The LMM showed an ability to understand the boundaries of geographic features and even infer terrain, for example, avoiding predicting locations in the ocean. This suggests a potential for AI to incorporate ecological information, like whether a species lives on land or in water, into its georeferencing.
However, the researchers noted some challenges, particularly with very long linear features like rivers or highways, which can make the map too broad. They also observed that current LMMs, not being specifically trained for map reading, sometimes struggled with the continuity of linear features or confused labels with actual feature locations.
Despite these minor limitations, the performance of this new multi-modal approach is a significant leap forward. The paper suggests that by fine-tuning LMMs with vast amounts of natural history records and their corresponding maps, we could develop even more powerful “GeoAI” models capable of truly comprehending cartographic information, further unlocking the potential of historical biological data.
