TLDR: The LONGQAEVAL research introduces a framework for reliably evaluating long-form clinical AI answers, addressing challenges like medical expertise and inconsistent human judgments. It compares coarse (answer-level) and fine-grained (sentence-level) annotations across correctness, relevance, and safety. Key findings include improved inter-annotator agreement for correctness with fine-grained methods, better agreement for relevance with coarse methods, and the efficiency of partial fine-grained annotations. The study also shows that fine-grained evaluations can reduce biases related to answer length and that LLMs perform comparably to physicians in correctness and relevance, though safety remains a concern. Additionally, LLMs can serve as effective judges for certain dimensions when given appropriate instructions.
Evaluating how well artificial intelligence systems answer complex medical questions is a challenging task. It requires deep medical knowledge, and even experts can sometimes disagree on what makes a good answer, especially when dealing with long, detailed responses. This makes it difficult and expensive to reliably assess the performance of new AI models in healthcare.
A new research paper introduces LONGQAEVAL, a comprehensive framework designed to make evaluating long-form clinical question-answering (QA) systems more reliable, especially when resources are limited and high medical expertise is essential. The framework offers a set of recommendations based on a study where physicians annotated 300 real patient questions, answered by both human doctors and large language models (LLMs).
Understanding Evaluation Approaches
The study compared two main ways of evaluating answers: coarse-grained and fine-grained. In a coarse-grained evaluation, annotators assess the entire answer as a whole. In contrast, fine-grained evaluation involves assessing individual sentences within the answer. The evaluations focused on three critical dimensions: correctness (accuracy of medical information), relevance (how well the answer addresses the specific question), and safety (whether the answer communicates potential risks or contraindications).
Key Findings on Reliability and Efficiency
The researchers found that the best evaluation approach depends on the dimension being assessed. For correctness, fine-grained annotation significantly improved agreement among annotators. This suggests that breaking down answers into individual sentences helps experts agree more on factual accuracy. However, for relevance, coarse-grained annotation led to better agreement, indicating that understanding the overall context is more important for judging how well an answer addresses the patient’s concern. Judgments on safety remained inconsistent across both methods, highlighting a persistent challenge in this critical area.
A particularly important finding for resource-constrained settings is that annotating only a small subset of sentences (e.g., three sentences per answer) can provide reliability comparable to coarse annotations. This approach significantly reduces the time and cost involved in evaluations without sacrificing quality.
Addressing Biases and LLM Performance
The study also revealed that fine-grained annotations can help mitigate biases related to the length of an answer. Longer answers, often generated by LLMs, can sometimes be perceived as more accurate even if their sentence-level accuracy is similar to shorter, physician-generated responses. By forcing annotators to evaluate sentence by sentence, fine-grained methods help ensure fairer comparisons between different systems.
When it comes to the performance of LLMs, the study found that models like GPT-4 and Llama-3.1-Instruct-405B provided information that was comparable to physician answers in terms of correctness and relevance for general primary care questions. In fact, expert annotators sometimes rated LLM outputs as more relevant, possibly due to their tendency to provide more context and background. However, similar to human doctors, LLMs still struggle with consistently providing satisfactory safety warnings, indicating an area for further improvement.
LLMs as Evaluators
The research also explored the potential of using LLMs themselves as judges. It found that when given fine-grained instructions, an LLM-as-judge (GPT-4o) could achieve agreement with human experts that was comparable to, or even exceeded, expert-expert agreement for correctness and relevance. This suggests that LLMs could potentially supplement human expert judgments in future evaluation studies, especially for factual dimensions.
Also Read:
- Moving Beyond Textbook Cases: A New Way to Evaluate AI in Medical Diagnosis
- Unpacking LLM Factual Consistency: Why Simple Answers Don’t Guarantee Complex Truths
Recommendations for Future Evaluations
Based on these findings, the paper offers several practical recommendations:
- Tailor your annotation design to the specific dimension you are evaluating: use fine-grained methods for factual aspects like correctness, and coarse methods for context-dependent aspects like relevance.
- Consider using partial fine-grained annotations (e.g., three sentences per answer) to reduce costs and effort while maintaining reliability.
- Prefer fine-grained annotations for generating system ratings and rankings, especially for correctness and safety, as they can help reduce biases related to answer length and presentation style.
- While LLMs show promise in providing correct and relevant clinical answers, their deployment in real-world patient care settings should proceed with caution and rigorous evaluation, particularly concerning safety.
The LONGQAEVAL framework provides valuable guidance for researchers and practitioners aiming to design more effective and efficient evaluation studies for long-form clinical QA systems. You can read the full research paper for more details here: LONGQAEVAL: Designing Reliable Evaluations of Long-Form Clinical QA under Resource Constraints.
