TLDR: A new research paper explores how large language models (LLMs) can reduce documentation burden for healthcare practitioners by structuring speech transcripts from nurse dictations and extracting medical orders from doctor-patient consultations. The study introduces two new open-source datasets, SYNUR and SIMORD, to facilitate research in these data-scarce areas. Evaluations show that both closed-weight and open-weight LLMs can achieve strong performance, demonstrating the viability of AI-driven solutions for real-world clinical tasks.
Healthcare professionals, particularly nurses and doctors, face a significant burden from documentation, which often takes time away from direct patient care. A new research paper explores how large language models (LLMs) can help alleviate this issue by transforming spoken clinical interactions into structured, usable data.
The paper, titled “Empowering Healthcare Practitioners with Language Models: Structuring Speech Transcripts in Two Real-World Clinical Applications,” investigates two critical yet underexplored areas in clinical natural language processing (NLP): creating structured reports from nurse dictations and extracting medical orders from doctor-patient consultations. These tasks are challenging due to the scarcity and sensitivity of clinical data.
Addressing the Documentation Challenge
The core idea is to use advanced AI, specifically large language models like GPT-4o, to automate the process of converting spoken words into organized information that can be easily integrated into electronic health records (EHRs). This could free up healthcare providers to focus more on their patients.
The researchers tackled two distinct problems:
-
Structured Reporting from Nurse Dictations: Nurses often dictate observations about patients. The goal here is to automatically extract key clinical observations and populate structured tables, known as flowsheets, in the EHR. This is complex because nurse dictations can be long, contain natural speech patterns like hesitations, and need to align with specific hospital-defined schemas.
-
Medical Order Extraction from Doctor-Patient Consultations: During consultations, doctors issue various medical orders (medications, labs, imaging, follow-ups). The challenge is to accurately identify these orders from the conversation, extract their descriptions, reasons, types, and even pinpoint where they were mentioned in the transcript.
New Datasets for Future Research
A major hurdle in these areas has been the lack of publicly available, high-quality clinical datasets. To address this, the paper introduces two new open-source datasets:
-
SYNUR (SYnthetic NURsing dataset): This dataset was created to help with nurse observation extraction. It involved a six-stage pipeline that generated realistic, non-sensitive synthetic nurse dictations. These synthetic notes were then meticulously reviewed and annotated by expert nurses to ensure their accuracy and resemblance to real-world data.
-
SIMORD (SIMulated ORDer dataset): This dataset focuses on medical order extraction. It was built by having medically trained annotators create gold-standard medical orders from high-quality doctor-patient conversations, simulating how doctors would document orders in an EHR.
Evaluating Language Models
The study evaluated both commercially available (closed-weight) and open-source (open-weight) LLMs on these tasks. For nurse observation extraction, models like GPT-4.1 and GPT-4o showed strong performance, with few-shot learning (providing a few examples to the model) significantly improving results. The SYNUR dataset proved valuable in this evaluation, despite some differences compared to proprietary hospital datasets.
For medical order extraction, various LLMs were tested. While no single model consistently outperformed others across all metrics, the study found that open-weight models like MediPhi-Instruct, a medical variant of Phi-3.5-mini-Instruct, achieved competitive results. Notably, MediPhi-Instruct reached parity with GPT-4o on some metrics when given two examples, demonstrating the potential of smaller, more accessible models for this task. The research also highlighted challenges such as models sometimes aggregating multiple orders into one or producing malformed outputs, especially for provenance information.
Also Read:
- AI Models Streamline Clinical Data Standardization with HL7 FHIR
- M3: Simplifying Clinical Data Access with Conversational AI
Looking Ahead
The findings suggest that LLMs can indeed play a crucial role in reducing the documentation burden in clinical settings, paving the way for more efficient healthcare workflows. While synthetic data like SYNUR helps overcome data scarcity, the paper acknowledges that it might not fully capture the entire complexity of real clinical language. Future work will likely involve leveraging more real nurse dictations and refining the extraction processes to handle nuanced clinical language and complex data structures.
For more in-depth information, you can read the full research paper available at arXiv.org.
