TLDR: SARHAchat is an LLM-based chatbot developed by UNC School of Nursing to provide reliable, user-centered sexual and reproductive health counseling. It addresses common AI challenges like hallucination and lack of specialized knowledge by using a five-stage conversational flow, a bi-level memory module for factual accuracy, and structured reasoning for personalized recommendations. Evaluations show SARHAchat significantly outperforms baseline models in medical safety and conversational quality, making it a promising tool for pre-clinical SRH care.
In the evolving landscape of healthcare technology, Artificial Intelligence (AI) is showing immense potential, particularly in areas like disease diagnosis and drug discovery. However, when it comes to sensitive and complex medical fields such as Sexual and Reproductive Health (SRH), existing AI conversational systems often face significant challenges. These systems frequently struggle with generating inaccurate information, a phenomenon known as hallucination, and often lack the specialized knowledge required for such delicate topics. Furthermore, many current AI applications in healthcare tend to focus more on diagnostic capabilities rather than providing comprehensive patient care and education.
Addressing these critical gaps, researchers at the UNC School of Nursing have introduced SARHAchat, a groundbreaking Large Language Model (LLM)-based chatbot. SARHAchat is designed as a reliable, user-centered system that seamlessly integrates medical expertise with empathetic communication to significantly enhance SRH care delivery. This innovative chatbot aims to provide accurate and contextually appropriate contraceptive counseling while maintaining a natural and supportive conversational flow.
What is SARHAchat and How Does It Work?
SARHAchat, which stands for Sexual And Reproductive Health Assistant chatbot, is an initiative focused on providing reliable AI-driven patient triaging and counseling for SRH topics. It specifically facilitates pre-clinical contraceptive care by gathering a user’s health history, delivering personalized counseling and education, and creating user profiles. These profiles are designed to streamline subsequent interactions with healthcare providers and simplify prescription processes.
The system operates through a predefined five-stage conversational flow, ensuring a structured yet flexible interaction:
- Stage 1: Initial Information Gathering – This involves understanding the user’s intent, gender, and previous experiences.
- Stage 2: Preference Screening – The chatbot inquires about user preferences regarding frequency and potential side effects of contraceptive methods.
- Stage 3: Health Screening – Critical medical history is collected to ensure safety and suitability of recommendations.
- Stage 4: Appropriate Contraception Recommendation – Based on the gathered information, SARHAchat provides personalized recommendations.
- Stage 5: Profile Verification and Summary Generation – The system verifies the user’s profile and generates a downloadable summary, which can be shared with a healthcare provider.
Key Technologies Behind SARHAchat
To overcome common LLM limitations like hallucination and context window constraints, SARHAchat incorporates several advanced techniques:
Bi-level Memory Module: This system uses both short-term and long-term memory. Short-term memory manages the immediate dialogue history, extracting user factors to query relevant information. External long-term memory is an expert-maintained key-value store containing distilled, up-to-date knowledge on contraceptive methods, including current CDC guidelines. This ensures factual accuracy and reliability.
Structured Reasoning and Guided Response Generation: Developed in collaboration with healthcare professionals, the chatbot employs a step-by-step reasoning chain that mimics expert clinical decision-making. It explicitly considers user preferences, medical history, and weighted decision criteria. A “thought injection” prompting technique integrates this reasoning chain and an explicit rationale into the final prompt, guiding the LLM to generate accurate, contextually appropriate, and justified recommendations in natural language, thereby significantly reducing the risk of hallucination.
The chatbot also supports a generative user interface that allows for seamless, multi-modal interaction, combining textual responses with visual aids and downloadable resources to enhance comprehension and information delivery.
Also Read:
- Crafting Reliable AI Clinical Reasoning: The Power of Expert-Curated Prompts in Fertility Treatment
- Guide-RAG: Optimizing AI Chatbot Responses for Long COVID Clinical Questions
Evaluation and Performance
SARHAchat was rigorously evaluated against a previous baseline version, which used a simpler prompting approach with ChatGPT. The evaluation focused on two crucial dimensions: medical safety and conversational quality. The results were compelling:
- Medical Safety: SARHAchat achieved a remarkable pass rate of 98.22%, significantly higher than the baseline’s 85.21%. It drastically reduced critical failures, with virtually no health screening omissions or recommendations of contraindicated methods. Critical information errors also decreased from 11 to just 3.
- Conversational Quality: The chatbot performed exceptionally well, receiving satisfactory ratings in 98.82% of conversations, compared to 89.35% for the baseline. This indicates superior interaction coherence and empathetic engagement.
These evaluations demonstrate SARHAchat’s ability to streamline SRH pre-clinical care and highlight its strong potential for clinical deployment with minimal technical barriers. For more detailed information, you can refer to the full research paper available here.
