TLDR: This paper from the Bridge2AI consortium outlines the critical importance of standardized metadata for making biomedical research data “AI-ready.” It details how four Grand Challenges within Bridge2AI are implementing various metadata standards, data models, and repositories to ensure data FAIRness, provenance, characterization, explainability, ethics, sustainability, and computability. The report highlights current practices, challenges, and future recommendations for harmonizing metadata across diverse biomedical datasets to accelerate AI/ML applications in health.
In the rapidly evolving landscape of artificial intelligence and machine learning, the quality and organization of data are paramount. This is especially true in biomedical research, where complex and sensitive information needs to be meticulously managed to unlock its full potential. A recent paper from the Bridge2AI consortium sheds light on the crucial role of metadata standards in making biomedical research data truly “AI-ready.”
The concept of AI-readiness refers to how well data can be optimally and ethically used for AI and machine learning methods, encompassing model training, data classification, and explainable predictions. The Bridge2AI consortium has established specific criteria for biomedical datasets to achieve this readiness, focusing on principles like FAIRness (Findability, Accessibility, Interoperability, and Reusability), provenance, characterization, explainability, sustainability, and computability, alongside robust ethical data practices.
Biomedical datasets present unique challenges. They are rarely perfect “ground truths,” often undergo extensive pre-processing, may involve human subjects with legal restrictions, and require special handling for pre-model explainability. Metadata, which is essentially “data about data,” becomes indispensable here. Without proper context or origin information, data loses its meaning, impacting its authenticity, consent, reproducibility, and ethical use. Therefore, metadata must be both machine-readable and human-readable, mapped to standardized vocabularies or ontologies to ensure interoperability.
The Bridge2AI initiative includes four data-generating projects, known as Grand Challenges (GCs), each focused on creating AI/ML-ready datasets to address complex biomedical and behavioral research problems. These GCs develop standardized, multimodal data, tools, and training resources while adhering to ethical data practices. Examples include using voice as a biomarker, building genomic tools, modeling disease trajectories, and mapping cellular health indicators.
Metadata Standards Across Bridge2AI Grand Challenges
The paper details the specific metadata standards and tools employed by each of the four GCs:
The AI/ML for Clinical Care GC, also known as Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, focuses on intensive care clinical data. It uses and extends version 5.4 of the OMOP Common Data Model (CDM) to unify diverse data types like medical waveforms (using Waveform Database Software Package) and images (using Digital Imaging and Communications in Medicine, DICOM). CHoRUS has even motivated additions to the OMOP CDM, including a new Vocabulary Metadata table, and is developing extensions to accommodate new data types. Data quality is rigorously assessed using standardized scripts, and integrated datasets are released as beta versions before full validation.
The Functional Genomics GC, or Cell Maps for AI (CM4AI), aims to map the spatiotemporal architecture of human cells. CM4AI packages its datasets using Research Object Crate (RO-Crate) with JSON-LD serialization, describing data with Schema.org, DataCite, Evidence Graph Ontology (EVI), JSON Schema, and Frictionless Data vocabularies. It assigns persistent identifiers (PIDs) to all datasets and software, ensuring provenance graphs are resolvable. Data dictionaries are provided as Frictionless Data schemas, defining dataset structure, column labels, datatypes, and mappings to vocabulary terms.
The Precision Public Health GC, known as Voice as a Biomarker of Health, investigates connections between human voice recordings and disease states. This GC organizes its datasets using a schema aligned with the Brain Imaging Data Structure (BIDS). Audio data is stored in WAV format, while clinical and phenotypic data are in tab-separated values (TSV) with JSON data dictionaries. They have also developed a Voice as a Biomarker for AI Health profile for FHIR R4 to ensure broader interoperability. Raw audio and questionnaire data are initially stored in a REDCap system before conversion to the BIDS structure.
The Salutogenesis GC, or Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights (AI-READi), creates an ethically-sourced dataset for Type 2 Diabetes patients. Its v2.0.0 dataset is organized using the Clinical Dataset Structure (CDS), inspired by BIDS. It incorporates various formats for different modalities: Waveform Database (WFDB) for cardiac data, OMOP CDM for clinical observations, Earth Science Data System (ESDS) for environmental data, DICOM for retinal imaging, and Open mHealth standards for wearable sensor data. Each dataset version released through the FAIRhub platform receives a unique DOI, linking to relevant IDs like ClinicalTrials.gov and ORCIDs.
Storing and Managing Metadata
Bridge2AI GCs utilize both NIH-approved domain-specific and generalist repositories for data storage. For data containing personal health information, strategies like date shifting, pixel scrubbing, and novel anonymization methods are employed to limit re-identification risks. The AI/ML for Clinical Care GC uses a secure enclave, while the Precision Public Health GC releases anonymized data on platforms like Health Data Nexus and Physionet. The Salutogenesis GC uses FAIRhub, which requires prospective users to agree to custom licenses and undergo identity verification. The Functional Genomics GC deposits data in repositories like NIH’s Sequence Read Archive (SRA), Figshare, and Dataverse, with cell map outputs also going to the Network Data Exchange (NDEx).
Also Read:
- Charting the Path to Self-Driving Science with AI Agents
- Unifying Autoformalization: A Framework for Bridging Informal and Formal AI Reasoning
Recommendations for AI-Ready Metadata
The Bridge2AI Standards Working Group has outlined seven minimum requirements for AI-ready metadata:
- FAIRness: Digital objects comply with the FAIR Principles.
- Provenance: Origins and transformational history are richly documented.
- Characterization: Content semantics, statistics, and standardization are well-described, including quality and bias.
- Pre-Model Explainability: Supports explainability of predictions based on data, fit for purpose, and data integrity.
- Ethics: Ethical data acquisition, management, and dissemination are documented.
- Sustainability: Digital objects and metadata are stored in FAIR, stable archives.
- Computability: Standardized, computationally accessible, portable, and contextualized.
The paper emphasizes that clinical metadata is crucial for providing context to raw observations, transforming them into meaningful information. Standardized terminologies (SNOMED-CT, LOINC, RxNorm, ICD-10) and data models (OMOP CDM, HL7 FHIR) are vital for consistency and interoperability. Large-scale initiatives like the All of Us Research Program and the National Clinical Cohort Collaborative (N3C) demonstrate the success of standardized frameworks in enhancing care quality while protecting privacy through tiered access controls and de-identification techniques.
Despite its importance, improper metadata management poses significant challenges, including data quality issues, inconsistent formats, and privacy risks if not properly secured. The paper advocates for meticulous attention to metadata to ensure data are accurate, standardized, and secure, making them truly “AI-ready.”
Looking ahead, Bridge2AI is actively working on refining and implementing standard data description templates like Datasheets, Healthsheets, and Croissant, which can be directly integrated into machine learning operations pipelines. Cross-GC metadata standardization is also a priority, aiming to leverage Large Language Models (LLMs) for generating rich metadata annotations and automated explanations for AI models. Integration with the Common Fund Data Ecosystem (CFDE) is another key next step, aiming to present a unified, searchable collection of Bridge2AI GC metadata while ensuring patient privacy.
This ongoing work within Bridge2AI is crucial for harmonizing diverse biomedical datasets and maximizing the benefits of standardization, ultimately enhancing the efficiency and effectiveness of AI-powered biomedical research. For more detailed information, you can refer to the full research paper here.
