TLDR: A new method called MUSE (Multi-LLM Uncertainty via Subset Ensembles) improves the reliability and accuracy of large language models by intelligently combining predictions from multiple models. It uses Jensen-Shannon Divergence to measure disagreement and entropy to estimate intrinsic noise, identifying and aggregating the most trustworthy subsets of LLMs. This leads to better uncertainty estimates and improved performance, especially in critical applications like healthcare, by balancing model diversity with reliability.
Large Language Models (LLMs) have become incredibly powerful tools, excelling in a vast array of natural language processing tasks. However, despite their impressive capabilities, LLMs often exhibit inconsistencies in their outputs. This variability highlights a crucial need: quantifying their uncertainty, especially when these models are deployed in high-stakes environments like healthcare, where reliability is paramount for trust, safety, and informed decision-making.
Traditional approaches to understanding and quantifying LLM uncertainty typically focus on individual models. This overlooks a significant opportunity: the potential of combining diverse models. Researchers hypothesize that different LLMs, due to variations in their training data, objectives, and architectures, often make complementary predictions. This means that by intelligently aggregating their outputs, we could achieve more reliable uncertainty estimates and, consequently, more trustworthy predictions.
To harness this potential, a new method called MUSE (Multi-LLM Uncertainty via Subset Ensembles) has been proposed. MUSE is a straightforward, information-theoretic approach designed to identify and combine well-calibrated subsets of LLMs. It leverages Jensen-Shannon Divergence (JSD) to measure the degree of disagreement among models, which serves as a signal of ‘epistemic uncertainty’ – uncertainty arising from incomplete knowledge. Alongside this, it considers ‘aleatoric uncertainty,’ which reflects the inherent ambiguity in the input data itself, by calculating the average entropy of individual model predictions.
The core idea behind MUSE is to select a group of models whose predictions show both low disagreement and low intrinsic uncertainty. This careful selection process helps to balance the benefits of diversity (which can introduce useful signals) with the need to avoid noise (which can degrade accuracy and calibration). MUSE offers two main strategies for selecting these subsets: a greedy version that starts with the most confident prediction and iteratively adds models that increase diversity within a tolerance, and a conservative version that prioritizes minimizing total uncertainty.
Once a suitable subset of LLMs is identified, MUSE aggregates their predictions. This can be done through a simple unweighted average or an ‘aleatoric-aware’ weighting, where predictions from more confident (low-entropy) models are given higher importance. This ensures that the final prediction is not just an average, but a carefully considered consensus from the most reliable models within the ensemble.
The effectiveness of MUSE was tested on several binary prediction datasets, including TruthfulQA (a general domain question-answering benchmark) and two clinical datasets, EHRShot and MIMIC-Extract, derived from real-world patient records. The results were compelling: MUSE consistently demonstrated improved calibration and predictive performance when compared to single-model baselines and even simple, naive ensemble methods.
For instance, while some single LLM methods might achieve high accuracy (AUROC), they often suffer from poor calibration, meaning their stated confidence doesn’t match their actual correctness. MUSE, however, provides a more balanced outcome, achieving comparable accuracy with significantly lower calibration error. This indicates that MUSE not only helps models predict better but also makes them ‘know what they don’t know’ more accurately.
A key strength of MUSE is its adaptive nature. The research showed that performance improves when ‘stronger’ LLMs (those with better individual performance) are present in the candidate pool. Conversely, if the pool is dominated by weaker or noisier models, the aggregated performance can degrade. This underscores that MUSE is not about simply adding more models, but about selectively combining those that contribute meaningful and calibrated signals to reduce overall uncertainty. For more technical details, you can refer to the full research paper here.
Also Read:
- Guiding Large Language Models Through Better Questioning
- Cautious Next Token Prediction: Enhancing LLM Reliability Through Adaptive Exploration
In conclusion, MUSE offers a valuable advancement in making LLMs more reliable and trustworthy, particularly for critical applications. By focusing on selective multi-model aggregation and an information-theoretic approach to uncertainty, it paves the way for more robust and well-calibrated AI systems. Future work aims to explore dynamic selection strategies and scale the method to even larger collections of LLMs for real-world deployment.
