TLDR: A study investigated the impact of free-text, structured, and AI-prefilled structured reporting on radiologists’ diagnostic accuracy, efficiency, and visual behavior in chest radiography. It found that AI-prefilled structured reporting significantly improved diagnostic accuracy for all readers, especially novices, and both structured and AI-prefilled reporting substantially reduced reporting times. Eye-tracking showed more efficient visual processing with structured and AI-assisted modes, guiding visual attention more effectively. User experience was generally positive for structured and AI-assisted reporting, though trust in AI remained a factor.
A recent study delves into how different reporting methods, including those enhanced by artificial intelligence, influence radiologists’ diagnostic accuracy, efficiency, and visual behavior when interpreting chest radiographs. The research, titled “Effect of Reporting Mode and Clinical Experience on Radiologists’ Gaze and Image Analysis Behavior in Chest Radiography,” was conducted by a team including Mahta Khoobi, Marc S. von der Stueck, Felix Barajas Ordonez, Anca-Maria Iancu, Eric N. Corban, Julia Nowak, Aleksandar Kargaliev, Valeria Perelygina, Anna-Sophie Schott, Daniel Pinto dos Santos, Christiane Kuhl, Daniel Truhn, Sven Nebelung, and Robert M. Siepmann.
The study aimed to understand the impact of various reporting modes – free-text (FT) reporting, structured reporting (SR), and AI-prefilled structured reporting (AI-SR) – on how radiologists interact with medical images. With the increasing demand for radiological services and the growing complexity of imaging, developing efficient and accurate reporting workflows is crucial.
Study Design and Methodology
Conducted from July to December 2024, the prospective study involved eight readers: four novice (pre-graduate medical students and residents without full radiography training) and four non-novice (residents with completed training but pending board certification). Each reader analyzed 35 bedside chest radiographs per session across three different reporting modes, with two-week washout periods between sessions to minimize learning effects.
The radiographs and reporting interface were presented on a screen-based eye-tracking system, which recorded eye movements. Key outcome measures included diagnostic accuracy (compared to a ground truth established by expert radiologists), reporting time per radiograph, various eye-tracking metrics (like fixation duration and saccade counts), and user experience feedback collected via questionnaires.
Key Findings
The study yielded several significant insights:
-
Diagnostic Accuracy: While free-text and structured reporting showed similar diagnostic accuracy (κ=0.58 and κ=0.60, respectively), AI-prefilled structured reporting significantly improved accuracy to κ=0.71. Novice readers experienced the most substantial gains in accuracy, reaching levels comparable to non-novice readers when using AI assistance.
-
Reporting Efficiency: Both structured reporting and AI-prefilled structured reporting dramatically reduced reporting times. Free-text reporting averaged 88 seconds per radiograph, which dropped to 37 seconds with structured reporting and further to 25 seconds with AI-prefilled structured reporting. Novice readers again saw the largest improvements in efficiency.
-
Image Analysis Behavior: Eye-tracking metrics revealed that structured reporting and AI-prefilled structured reporting led to more efficient visual processing, characterized by decreased fixation durations and saccade counts. For novice readers, structured reporting shifted their gaze focus from the report display field to the radiograph display field. Non-novice readers, however, maintained their visual focus on the radiograph regardless of the reporting mode. Visual attention became more concentrated on key anatomical regions with structured and AI-prefilled structured reporting.
-
User Experience: Structured reporting and AI-prefilled structured reporting were generally preferred by readers in terms of user satisfaction, interface convenience, perceived efficiency, and willingness for clinical adoption. While the usefulness of AI was rated highly, trust in AI suggestions remained limited among the participants.
Also Read:
- Exploring Foundation Models in Medical Imaging: A Comprehensive Review
- Advancing Medical Diagnostics with Real-Time Deep Learning for Image Analysis
Conclusion
The research concludes that structured reporting significantly enhances efficiency by guiding visual attention more effectively toward the image, particularly benefiting less experienced readers. Furthermore, AI-prefilled structured reporting demonstrably improves diagnostic accuracy. These findings suggest that integrating structured reporting and AI assistance into radiological workflows can lead to more efficient and accurate diagnoses, though further research is needed to address limitations such as sample size, experimental setting, and the generalizability of findings to more experienced radiologists.
For more detailed information, you can read the full research paper here.
