TLDR: This research introduces a dual-stage, lightweight system for summarizing patient charts for emergency physicians. Running entirely offline on two NVIDIA Jetson Nano devices, it first retrieves relevant EHR sections and then generates a two-part summary (critical findings and context-specific narrative) using a small language model. The system prioritizes patient privacy, low latency, and cost-effectiveness, and uses an LLM-as-Judge framework to ensure factual accuracy. It demonstrates effective summary generation in under 30 seconds, significantly improving efficiency and data security in emergency care.
Emergency physicians often face a critical challenge: sifting through vast amounts of unstructured patient data in electronic health records (EHRs) to find vital information quickly. This time-consuming task can be particularly difficult for elderly patients with complex medical histories and multiple prior visits. To address this, a new system has been developed that automates the summarization of patient charts, aiming to provide physicians with key information and context-specific insights rapidly, leading to faster and more accurate decision-making.
Traditional approaches to clinical summarization often rely on large, cloud-based language models. However, patient health information is highly sensitive, and privacy regulations frequently prohibit sending EHR data to external cloud services. Furthermore, internet connectivity can be unreliable in emergency settings like ambulances, rural clinics, or disaster zones. These limitations highlight the need for solutions that can operate locally, directly at the point of care.
An Offline, Dual-Stage Summarization System
Researchers have introduced a novel, fully offline, edge-resident EHR summarization system specifically designed for emergency medicine. This innovative approach uses a dual-device architecture, leveraging the power of small language models (SLMs) on resource-constrained hardware to ensure patient privacy and reduce dependency on internet connectivity. The system is implemented using two NVIDIA Jetson Orin Nano boards, which are inexpensive and energy-efficient IoT devices.
The process is divided into two stages:
- Nano-R (Retrieve): The first Jetson Nano handles information retrieval. It stores locally all patient EHRs, splits long notes into semantically coherent sections, and then uses an embedding model to search for the most relevant sections based on a clinician’s query (e.g., “chest pain”). This significantly narrows down the input for the summarization stage.
- Nano-S (Summarize): The second Jetson Nano receives the retrieved context from Nano-R via a lightweight socket link. It then runs a locally hosted small language model to generate the summary. This dual setup avoids the overhead of swapping models in and out of a single device’s memory and allows for parallel processing, substantially reducing overall processing time.
Tailored Summaries for Emergency Care
The system produces a two-part summary output, designed to meet the specific needs of emergency physicians:
- Critical Findings: A concise, bulleted list of three “need-to-know” facts about the patient, such as ongoing or recurring issues, that are universally relevant.
- Context-Specific Summary: A paragraph tailored to the patient’s chief complaint, highlighting relevant demographics, medications, allergies, conditions, recent events, and surgical history from the chart.
This structured approach ensures that physicians can quickly identify both general must-know background information and pertinent details for the current issue, aligning with how clinicians prioritize information in emergencies.
Ensuring Summary Quality and Reliability
Given the critical nature of clinical information, factual accuracy is paramount. The system employs a unique evaluation mechanism called the Factual Accuracy (FA) score, which uses an LLM-as-Judge framework. Instead of relying on traditional metrics that compare summaries to a “gold standard” (which doesn’t exist in this context), the FA score verifies each fact in the generated summary against the original EHR content. Claims are categorized as Supported, Contradicted, or Not Found, with contradicted claims incurring a strong penalty. This method prioritizes factual faithfulness and task-oriented utility.
Also Read:
- Generalist AI Models Show Promise in Emergency Department Decision Support
- MedReflect: Enabling Medical LLMs to Learn Self-Correction Through Physician-Like Reflection
Performance and Practicality
Experiments were conducted using both the MIMIC-IV-Note dataset and de-identified real-world EHRs. Several small language models (under 7 billion parameters) were benchmarked, including Starling-LM, Neural-Chat, Mistral, and OpenChat, along with various prompting techniques like zero-shot and Chain-of-Thought. The results demonstrated that the fully offline system can effectively produce useful summaries, with end-to-end summarization taking under 30 seconds for smaller models. An ablation study confirmed that the dual-device setup significantly reduced latency and improved system stability compared to a single-device configuration.
This research represents a significant step towards bringing AI-powered clinical summarization directly to the point of care, enhancing patient privacy, and improving efficiency for emergency physicians. For more details, you can read the full paper here.
