TLDR: A new multi-agent AI system, featuring RecipientAgent, InquirerAgent, and DepartmentAgent, collaborates to enhance medical triage. It transforms unstructured patient symptoms into standardized records, asks targeted questions, and recommends appropriate hospital departments with high accuracy. The system addresses challenges like AI ‘hallucinations,’ diverse hospital structures, and inefficient questioning, demonstrating significant improvements in classification accuracy and adaptability through its intelligent guidance mechanisms and a comprehensive Chinese medical triage dataset.
The global healthcare system is currently facing immense pressure, largely due to the increased demand for medical services after the pandemic and a significant shortage of nursing staff. This situation has put a strain on emergency department triage systems, leading to longer wait times and patients leaving without being seen. To address these critical issues, researchers are exploring innovative AI-driven solutions.
While AI-based triage systems show promise, existing models often encounter several challenges. These include a lack of specialized medical knowledge, which can lead to incorrect classifications (often referred to as ‘hallucinations’ in AI), difficulty adapting to the varied department structures across different hospitals, and a tendency for AI to ask overly detailed questions that slow down the triage process.
A new multi-agent intelligent system has been developed to tackle these challenges. This system employs three specialized AI agents that work together to transform unstructured patient symptoms into accurate department recommendations. The core idea is to enhance specialized capabilities and reduce errors through a collaborative approach.
How the Multi-Agent System Works
The system is built around three main agents:
- RecipientAgent: This agent acts as a data processor. It takes a patient’s initial, often unstructured, symptom description and converts it into a standardized ‘History of Present Illness’ (HPI) record. This ensures that all relevant information, both explicit and implicit from previous questions, is organized clearly.
- InquirerAgent: Once the HPI is created, this agent identifies any missing critical information. It then generates targeted questions for the patient, ensuring that no information is repeated and that the questions are focused on clarifying details essential for accurate triage.
- DepartmentAgent: This is the decision-making core. Based on the complete HPI, a list of available departments, and dynamic guidance rules, this agent recommends the most appropriate primary and secondary departments. It avoids getting bogged down in minor symptom details, instead focusing on broader factors like treatment approaches, patient age, and gender to ensure efficient and accurate recommendations.
These agents collaborate through multiple rounds of interaction. If the system hasn’t reached a confident decision, the InquirerAgent and DepartmentAgent form a feedback loop, continuously refining information collection and department differentiation until an optimal triage result is achieved.
Intelligent Guidance Mechanisms
To further enhance accuracy and efficiency, the system incorporates two key guidance mechanisms:
- Inquiry Guidance: This mechanism uses department-specific rule libraries to guide the InquirerAgent. It helps identify crucial differential questions (e.g., for internal medicine, asking about chronic diseases) and also specifies what details to avoid (e.g., not getting too specific about pain locations, as the goal is triage, not full diagnosis). It also helps exclude certain specialties based on patient information.
- Classification Guidance: This mechanism, based on a rule engine, helps the DepartmentAgent make precise recommendations. It defines detailed comparison rules between departments, considering symptom characteristics, surgical indications, and medication needs. These rules have priority levels, allowing the system to dynamically adjust its logic, for example, by excluding surgical departments if there’s no trauma history.
Comprehensive Evaluation and Results
To ensure robust evaluation, the researchers built a comprehensive Chinese medical triage dataset from iiyi.com, comprising 3,360 real-world cases across 9 primary and 62 secondary departments. This diverse dataset helps the system adapt to various hospital requirements. The system also uses large language models to fill in missing information in incomplete medical records, a common issue with real-world data.
After four rounds of simulated doctor-patient interaction, the system achieved an overall accuracy of 74.2%. Specifically, it reached 89.2% accuracy in primary department classification and 73.9% accuracy in secondary department classification. The results showed a clear learning trend, with accuracy steadily improving as the conversation progressed, demonstrating the effectiveness of the multi-turn dynamic inquiry mechanism.
Beyond just accuracy, the system was evaluated across six clinical dimensions, including clinical inquiry capability, diagnostic reasoning, communication expression, and overall professionalism, achieving a robust average score. While strong in triage accuracy and communication, areas like diagnostic reasoning were identified for future improvement.
Error analysis revealed that the system’s errors tended towards lower clinical risk, with more mistakes in secondary department classification than in primary. It performed very well in high-sample departments like Internal Medicine and Obstetrics & Gynecology, while departments with fewer samples, such as Pediatrics and Oncology, showed areas for improvement.
Also Read:
- Smarter LLM Teams for Better Medical Decisions
- Automating Multi-Agent System Design with Finite State Machines: Introducing MetaAgent
Why It Works: Ablation Studies
Experiments confirmed the critical role of the RecipientAgent in structuring information. Models without this agent struggled significantly, highlighting that high-quality, structured input is essential for the system’s ability to reason and learn. The intelligent guidance mechanisms were also proven vital; the full system, leveraging both dynamic and comparison rules, achieved perfect accuracy in challenging cases, demonstrating superior learning efficiency compared to models without these guidance strategies.
This work provides a scalable framework for deploying AI-assisted triage systems that can accommodate the organizational diversity of healthcare institutions while ensuring clinically sound decision-making. For more details, you can refer to the full research paper: Collaborative Medical Triage under Uncertainty: A Multi-Agent Dynamic Matching Approach.
