TLDR: A new collaborative framework uses Large Multimodal Models (LMMs) to accurately forecast kidney health decline (eGFR) while providing clinically meaningful and interpretable explanations. It enhances open-source LMMs through visual knowledge transfer from stronger models, a short-term memory for consistent reasoning, and abductive reasoning for data-driven and hypothesis-based explanations, addressing challenges of privacy, cost, and interpretability in healthcare AI.
Chronic Kidney Disease (CKD) is a significant global health issue, and accurately predicting the estimated Glomerular Filtration Rate (eGFR) is crucial for managing the disease and making informed clinical decisions. While traditional methods exist, and even recent machine learning models have shown promise, a major hurdle for their adoption in healthcare settings has been the lack of interpretability. Clinicians need to understand how a prediction is made to trust the system and identify potential errors. This is where Large Multimodal Models (LMMs), which can process both visual and textual information, show great potential.
However, deploying powerful LMMs, especially proprietary ones, comes with challenges like high costs and data privacy concerns. On the other hand, open-source LMMs, while more suitable for local deployment, often struggle with complex clinical reasoning and interpreting visual data, sometimes leading to unreliable outputs. Addressing these issues, a new study introduces a collaborative framework designed to enhance the performance of open-source LMMs for eGFR forecasting, all while generating explanations that are meaningful to clinicians.
A Collaborative Approach to Kidney Health Prediction
The proposed framework, detailed in the research paper “Towards Interpretable Renal Health Decline Forecasting via Multi-LMM Collaborative Reasoning Framework”, operates in two main stages: image interpretation and eGFR prediction with explanation. The core idea is to enable open-source LMMs to learn from more capable models and to incorporate mechanisms that improve their reasoning and consistency over time.
In the first stage, a patient’s historical eGFR measurements, which are sensitive data, are transformed into de-identified trend line charts to protect privacy. These charts are then fed into a proprietary vision-language model, referred to as the “Teacher LMM.” This Teacher LMM extracts important clinical trends and summarizes the patient’s kidney function status from the visual data. The interpretations generated by the Teacher LMM are then evaluated for clinical accuracy and coherence, with the best interpretation selected to serve as external knowledge for the next stage.
The second stage involves an open-source “Student LMM,” deployed locally. This Student LMM receives multiple inputs: the patient’s eGFR trajectory in chart format, structured clinical and laboratory variables, and the interpretations generated by the Teacher LMM. To improve its reasoning and reduce the chance of generating incorrect information, the Student LMM uses a “Chain-of-Thought” prompting strategy. It first predicts the next eGFR value and then generates an explanation based on that prediction, ensuring the explanation is logically connected to the outcome.
Enhancing Consistency and Interpretability
A crucial aspect of this framework is the incorporation of a “short-term memory” mechanism. In sequential clinical tasks, maintaining context over time is vital. After each prediction, the prompt, predicted value, and explanation are stored. When the model moves to the next step, it retrieves this memory, along with the actual outcome of the previous prediction. This allows the model to self-correct and refine its reasoning, improving consistency and accuracy by recognizing evolving patterns.
To make the predictions truly interpretable, the framework employs two forms of “abductive reasoning.” “Selective abduction” grounds predictions in observable clinical data, such as eGFR trends or specific lab results like BUN (blood urea nitrogen) and UACR (urine albumin creatinine ratio). “Creative abduction” goes a step further by hypothesizing plausible but unobserved factors that could contribute to the predicted outcome. By combining data-driven and hypothesis-based reasoning, the model provides more comprehensive interpretations, which can also serve as a valuable educational tool for medical students and junior clinicians.
Also Read:
- Bridging AI Accuracy and Transparency in Medical Diagnosis
- Smarter LLM Teams for Better Medical Decisions
Experimental Insights and Future Potential
The researchers tested their method using data from the Kaohsiung Medical University Research database, focusing on a subset of 570 observations from 50 patients. They compared their framework against traditional machine learning models like Random Forest (RF) and a one-dimensional convolutional neural network (1D-CNN), as well as various other LMMs, both open-source and proprietary.
While the Random Forest model achieved the best overall predictive performance, it lacked the crucial interpretability that the new framework provides. The experiments showed that open-source LMMs like Llama 3.2 vision and Gemma 3.0 significantly improved their eGFR prediction accuracy when both the knowledge transfer and short-term memory mechanisms were applied, achieving performance comparable to proprietary models. Interestingly, Qwen 2.5 vision 32B performed very well even without these additional techniques, suggesting its native architecture might have stronger visual-language integration.
This modular framework offers flexibility, allowing different components to be combined based on a model’s capabilities. Its ability to enhance weaker models without extensive retraining makes it accessible for deployment in environments with limited resources. Beyond just prediction accuracy, the structured explanations generated by the framework hold significant value for real-world applications, such as aiding clinical decision support and serving as a training tool in medical education.
Although the framework was developed and evaluated using a single dataset, and the practical value of its explanations needs further validation by medical experts, its modular design and integration of reasoning-based explanations show clear potential for broader use in healthcare AI systems that prioritize both accuracy and transparency.
