TLDR: Researchers have developed an Epidemiological Knowledge Graph (eKG) by using an ensemble of Large Language Models (LLMs) to extract critical, structured information from the World Health Organization’s Disease Outbreak News (DONs). This daily-updated dataset and knowledge graph make previously underutilized narrative reports accessible for systematic analysis, enhancing global disease surveillance, research, and public health response. The AI-driven approach significantly improves data extraction accuracy and provides accessible interfaces for exploring the structured information.
In an era where artificial intelligence is rapidly advancing and social media provides a wealth of information, the field of epidemiology and public health research is undergoing a significant transformation. A new research paper introduces an innovative approach to leverage this technological shift, focusing on the World Health Organization’s (WHO) Disease Outbreak News (DONs).
For years, the WHO’s DONs have been a crucial source of information on global health emergencies, reporting on outbreaks since 1996. However, their narrative, unstructured format made systematic analysis and integration into automated health systems challenging, leading to their underutilization in academic research.
To overcome this, researchers have developed an Epidemiological Knowledge Graph, or eKG, by employing an ensemble of advanced generative AI techniques, specifically multiple Large Language Models (LLMs). This method extracts vital epidemiological information such as disease and pathogen names, involved countries, event dates, case totals, and mortality figures from the DONs. This extracted data is then made publicly available in a daily-updated dataset and the eKG, adhering to FAIR (Findable, Accessible, Interoperable, and Reusable) principles.
The process involves a daily cycle where new reports from WHO DONs are extracted, transformed, and loaded. These reports are then processed by an ensemble of LLMs, including Mistral-7B-OpenOrca, Zephyr-7B-Beta, and Meta-Llama-3-70B-Instruct. This ensemble approach uses a majority voting system to determine the most accurate extraction for each piece of information, even handling synonyms for disease names and countries to ensure consistency.
The creation of eKG is driven by broader goals: to quickly identify emerging health threats, foster more effective international collaborations, and accelerate the development of public health interventions. By structuring this data, the aim is to contribute to global efforts to prevent outbreaks from escalating into epidemics or pandemics, especially during the critical early stages when timely and accurate information is paramount for policymakers.
The eKG is built using Semantic Web technologies like RDF and OWL, making it accessible and interpretable by both machines and humans. It integrates existing ontologies like the Infectious Disease Ontology (IDO) and GeoNames for geographical entities, ensuring a robust and standardized framework.
Access to this valuable resource is provided through various services and interfaces. A SPARQL endpoint allows programmatic querying of the eKG, enabling researchers and developers to retrieve analytical information on disease outbreaks. Additionally, web interfaces like the Virtuoso Faceted Browser, LodView, and LodLive offer intuitive ways to visualize and navigate the knowledge graph, making the data more user-friendly.
The technical validation of this approach demonstrated that the ensemble of LLMs significantly outperforms individual models and even competes with commercial AI solutions in extracting epidemiological information. This robust performance ensures the reliability and quality of the eKG dataset, making it a powerful tool for epidemic outbreak modeling and surveillance systems.
Also Read:
- AI Agents That Understand Their Own Limits in Complex Data
- REx: Crafting Scientifically Valid Explanations for Drug Repurposing with Knowledge Graphs
This work represents a significant contribution to epidemiology by providing a comprehensive and structured dataset derived from previously unstructured WHO DONs. The application of AI to process and analyze this data marks a transformative step towards enhancing our global understanding and management of disease outbreaks. For more details, you can refer to the original research paper.
