TLDR: RawMed is a novel framework that generates highly realistic, multi-table, time-series Electronic Health Records (EHR) data. Unlike previous methods that require extensive preprocessing and feature selection, RawMed uses a text-based approach with data compression and temporal modeling to synthesize raw EHRs, preserving all columns and original values. It also introduces a new evaluation framework and demonstrates superior performance in data fidelity, clinical utility, and privacy preservation on open-source datasets.
Electronic Health Records (EHR) are a cornerstone of modern healthcare, meticulously documenting patient interactions and medical events over time. These records, often stored as complex, multi-table relational databases, are invaluable for healthcare research and the development of medical AI applications. However, the sensitive nature of patient data, coupled with strict privacy regulations, severely restricts the sharing and utilization of real EHR datasets.
To address this critical challenge, the concept of synthetic EHR datasets has emerged as a promising solution. Synthetic data offers a privacy-preserving alternative that can still support vital research and clinical applications. While previous methods for generating synthetic EHRs typically focused on creating medical records with a limited, expert-chosen set of features—often just a few vital signs or structured codes—these approaches often fall short in capturing the full complexity and richness of raw EHRs.
Introducing RawMed: A New Era for Synthetic EHR Generation
A groundbreaking new framework called RawMed is changing the landscape of synthetic EHR generation. RawMed stands out as the first framework designed to synthesize multi-table, time-series EHR data that closely mirrors raw, real-world EHRs. This innovative approach minimizes the need for extensive preprocessing, a common bottleneck in traditional methods, and instead leverages text-based representation and compression techniques to capture the intricate structures and temporal dynamics inherent in medical records.
One of RawMed’s core innovations is its ability to treat EHR data as text. This allows the framework to retain original values without the need for distorting transformations like numerical binning or one-hot encoding. This text-based approach also makes RawMed highly adaptable to various EHR datasets, regardless of their specific database schemas. To manage the increased data length that comes with textual representation, RawMed employs a technique called Residual Quantization. This effectively compresses the textualized time-series data into a more manageable latent space, significantly reducing computational complexity and enabling the system to handle vast multi-table EHR datasets with numerous columns and time steps.
For modeling the temporal relationships between events, RawMed utilizes a Transformer-based model called TempoTransformer. This model autoregressively predicts each token in a patient’s trajectory, incorporating explicit time tokens to provide crucial temporal context alongside event content. The framework also includes robust postprocessing steps to ensure that the generated synthetic data maintains structural integrity and semantic consistency, correcting any minor errors that might arise during the generation process.
Also Read:
- Unlocking Complex Data: A New Framework for Generating Realistic Synthetic Relational Tables
- Advancing Medical Video Generation with MedGen and MedVideoCap-55K
Comprehensive Evaluation and Promising Results
Recognizing the unique challenges in evaluating multi-table, time-series synthetic EHRs, RawMed introduces a novel and comprehensive evaluation framework. This framework goes beyond traditional single-table assessments, evaluating aspects such as distributional similarity, inter-table relationships, temporal dynamics, and privacy protection. Metrics include Column-wise Density Estimation (CDE) and Pairwise Column Correlation (PCC) for data fidelity, and new temporal metrics like Time Gap and Next Event Prediction to assess how well the synthetic data captures the timing and sequence of clinical events. Privacy is evaluated using Membership Inference Attacks (MIA).
In rigorous tests on two widely used open-source EHR datasets, MIMIC-IV and eICU, RawMed consistently outperformed existing baseline models in terms of fidelity and utility. It demonstrated superior performance in replicating inter-event interval distributions and event counts, and proved highly effective in downstream clinical prediction tasks, with performance closely matching that of real data. Furthermore, RawMed exhibited minimal vulnerability to membership inference attacks, confirming its robust privacy preservation capabilities.
The research also included ablation studies, which highlighted the critical role of each component within RawMed, particularly Residual Quantization and the explicit temporal modeling techniques. These studies confirmed that RawMed’s design choices are essential for its high performance in generating complex, time-series EHR data. The framework also showed promising scalability across different observation windows, indicating its potential for broader application.
RawMed represents a significant leap forward in the generation of synthetic EHR data. By producing high-fidelity, multi-table, time-series EHRs with minimal preprocessing, it empowers hospitals to share realistic datasets while safeguarding patient privacy. This innovation promises to accelerate medical AI research by providing researchers with flexible and realistic data for a wide array of downstream tasks. The code for RawMed is openly available for further research and development. You can explore the full research paper for more details. Read the full research paper here.
