TLDR: A new method called Conformal Information Pursuit (C-IP) helps Large Language Models (LLMs) ask more effective questions in interactive conversations. Unlike traditional methods that struggle with LLM’s often inaccurate confidence, C-IP uses “conformal prediction sets” to reliably measure uncertainty. This leads to LLMs asking fewer, more informative questions, improving their accuracy in tasks like 20 Questions and medical diagnosis.
Large Language Models (LLMs) are increasingly used for interactive question-answering, where they sequentially ask for information to arrive at a prediction. However, a significant challenge arises because LLMs often provide over- or under-confident probabilities for their outputs. This miscalibration makes it difficult to accurately estimate uncertainty, leading to suboptimal question selection and less efficient conversations.
Traditional methods, such as Information Pursuit (IP), aim to minimize the number of questions by selecting queries that maximize information gain or minimize uncertainty at each step. But when LLM probabilities are unreliable, these uncertainty estimates become inaccurate, hindering the model’s ability to ask the most informative questions.
To address this, researchers have proposed Conformal Information Pursuit (C-IP), a novel approach that leverages ‘conformal prediction sets’ to measure uncertainty. Unlike traditional conditional entropy, conformal prediction sets offer a robust and distribution-free way to estimate how uncertain an LLM is about its prediction. C-IP utilizes a mathematical relationship between these prediction sets and conditional entropy, allowing it to estimate uncertainty based on the average size of these sets. Essentially, the smaller the prediction set, the more confident the model is.
C-IP works by greedily selecting the next question that is expected to minimize the size of the prediction set. This means the model is guided to ask questions that will most effectively narrow down the possibilities and reduce its uncertainty. The paper explores two ways to construct these prediction sets: by uniformly sampling historical query patterns or by simulating query patterns directly from LLM interactions.
Also Read:
- Enhancing LLM Confidence Estimates: The Role of Data-Agnostic Features in Generalization
- Conversation Forests: A New Approach to Training LLMs for Complex Dialogues
Real-World Applications and Performance
The effectiveness of C-IP was demonstrated through experiments on two distinct tasks. First, in a game of 20 Questions, C-IP showed superior predictive performance and achieved correct answers with shorter sequences of questions compared to previous IP methods and other uncertainty-based approaches. This held true for both pre-defined (closed) and free-form (open) question sets.
Second, C-IP was applied to an interactive medical question-answering task using the MediQ dataset, which simulates a conversation between a doctor LLM and a patient LLM. In this complex setting, C-IP achieved competitive performance with direct, single-turn predictions (where all information is given at once), while also providing greater interpretability of the diagnostic process. It consistently outperformed traditional IP in specialties like Internal Medicine and Pediatrics, indicating its ability to select more informative queries during the interactive diagnosis.
This research highlights that using prediction set sizes is an effective way to measure uncertainty in LLMs, leading to more efficient and accurate interactive AI systems. For more details, you can read the full research paper here.
