TLDR: This research paper provides a comprehensive survey of how deep learning and large language models (LLMs) are being used to model Electronic Health Records (EHRs). It introduces a unified taxonomy covering data handling, neural network architectures, learning strategies, multimodal data integration, and LLM-based systems. The survey highlights the evolution from traditional deep learning to pretraining and LLM-driven approaches, discusses their diverse applications in clinical settings, and identifies key challenges such as data collection, model evaluation, and ensuring clinical alignment and explainability.
Electronic Health Records, or EHRs, are the digital backbone of modern healthcare, serving as comprehensive patient data repositories. They contain a vast array of information, from demographics and lab results to diagnoses, treatments, and clinical notes. However, this data is incredibly complex—it’s diverse, often collected at irregular intervals, and highly specific to medical contexts. These characteristics make it challenging for artificial intelligence (AI) to effectively analyze and model.
A recent survey delves into the significant advancements at the intersection of deep learning, large language models (LLMs), and EHR modeling. It provides a structured overview, categorizing methods across five key areas: how data is handled, the design of neural networks, learning strategies, combining different types of data (multimodal learning), and systems built on LLMs.
The Evolution of AI in EHR Modeling
Early AI approaches for EHRs relied heavily on ‘feature engineering,’ where human experts manually defined and extracted important information from raw data. With the rise of neural networks, the focus shifted to ‘architecture engineering,’ where models learned to identify relevant features themselves. Deep learning models, like Transformers, became crucial for understanding the temporal sequences in patient journeys.
A pivotal shift occurred from 2020 to 2023 with the adoption of ‘pretraining-based models.’ Models like BEHRT, MedBERT, and ClinicalBERT, built on Transformer architectures, were trained on vast amounts of clinical data using self-supervised learning. This moved the research focus from designing network structures to developing effective pretraining objectives.
More recently, large-scale foundation models, such as GatorTron, ClinicalT5, and MedPaLM, have emerged. These LLMs, with billions of parameters, are pretrained on diverse clinical texts, including EHRs and biomedical literature. They can support a wide range of clinical tasks through ‘prompting,’ pushing EHR modeling towards more general-purpose clinical reasoning. While prompting offers flexibility, fine-tuning these models remains important for specific tasks.
Key Design Dimensions in EHR Modeling
The survey highlights several crucial aspects of designing AI for EHRs:
-
Data-Centric Approaches: These methods focus on improving the quality and quantity of the training data itself. This includes selecting high-quality samples, transforming input data to enhance patterns, integrating information from multiple sources, and leveraging knowledge graphs to enrich data with medical context. Techniques like generating synthetic EHRs are also explored to increase data availability while preserving privacy.
-
Neural Architecture Design: This involves creating specialized network structures to handle the unique properties of EHR data. This includes ‘feature-aware’ modules that preprocess diverse data types (like numerical values or categorical codes), ‘structure-aware’ designs that leverage hierarchical coding systems (like tree-based or graph-based models), and ‘temporal modeling’ strategies that account for irregular and multi-timescale clinical events.
-
Learning-Focused Strategies: These define how models learn from data. Self-supervised learning, for instance, allows models to learn from unlabeled EHRs by predicting masked information or contrasting different views of the same data. Clustering methods help identify distinct patient subgroups, while continual learning enables models to adapt to new data over time without forgetting old knowledge.
-
Multimodal Learning: Modern healthcare often involves integrating various data types, such as medical images (X-rays, CT scans) and clinical text. Multimodal AI aims to align these heterogeneous sources to provide a more comprehensive patient understanding. This includes fine-grained alignment between image regions and text, and data-efficient methods for training models when paired image-text data is scarce.
-
LLM-Based Modeling Systems: This dimension covers how LLMs are adapted for EHR tasks. ‘Prompt engineering’ involves crafting specific instructions to guide LLMs for tasks like feature generation or clinical reasoning. ‘Pretraining and fine-tuning’ adapt general LLMs to medical domains. ‘Retrieval-augmented generation’ (RAG) enhances LLMs by allowing them to retrieve relevant information from external knowledge sources or patient records, improving factual consistency. Furthermore, ‘LLM-driven medical agents’ are emerging, designed to simulate complex clinical workflows, from diagnosis to treatment planning, by using memory, planning, and tool-based execution.
Also Read:
- Unlocking Brain Signals: A Review of Early EEG Foundation Models
- Aligning Language Models: The Power of Inferring Rewards with Inverse Reinforcement Learning
Clinical Applications and Future Directions
AI in EHRs has a wide array of applications across the patient care timeline. This includes understanding clinical documents (summarization, note generation, concept extraction, coding automation), supporting clinical reasoning and decision-making (diagnosis prediction, prognostic forecasting, cohort discovery), and optimizing clinical operations (triage, referral recommendations, patient-trial matching).
The survey highlights several emerging trends, such as the development of specialized foundation models for EHRs and multimodal clinical data, and the rise of clinical agents that can emulate human reasoning and interact with healthcare environments. However, significant challenges remain, particularly in establishing robust benchmarks and validation practices, developing models that truly capture the complex structured and temporal nature of EHRs, and ensuring AI systems are explainable and align seamlessly with real-world clinical workflows and guidelines.
For a comprehensive list of EHR-related methods, kindly refer to the companion website mentioned in the paper: https://survey-on-tabular-data.github.io/
