TLDR: A new research paper introduces Collaborative Decoding (CoDe), a novel method designed to resolve the long-standing trade-off between factual accuracy (faithfulness) and natural, engaging language (expressiveness) in Large Language Models (LLMs). CoDe achieves this by dynamically integrating output probabilities generated with and without external knowledge, guided by distribution divergence and model confidence. It also features a knowledge-aware reranking mechanism to prevent over-reliance on internal parametric knowledge. Experiments show CoDe significantly improves faithfulness without sacrificing expressiveness across various LLMs and tasks, offering a plug-and-play solution for more reliable and human-like AI responses.
Large Language Models (LLMs) have made incredible strides in generating human-like text, but they often face a significant challenge: balancing factual accuracy with engaging, natural language. This dilemma, known as the faithfulness-expressiveness trade-off, means that LLMs either produce responses that are factually correct but sound stiff and uninspired, or they generate creative text that might contain inaccuracies or ‘hallucinations’.
Current methods struggle to overcome this. When LLMs are heavily reliant on external knowledge to ensure faithfulness, they tend to copy information directly, leading to dull and repetitive responses. Conversely, when they prioritize expressiveness, they might deviate from the provided facts, resulting in ungrounded or incorrect statements. This trade-off is particularly evident in how different decoding strategies work: deterministic methods offer high fidelity but low expressiveness, while stochastic methods boost diversity at the cost of accuracy.
Introducing Collaborative Decoding (CoDe)
To address this fundamental limitation, researchers Chenxu Yang, Qingyi Si, and Zheng Lin have introduced a novel approach called Collaborative Decoding (CoDe). This method aims to dynamically integrate the model’s internal knowledge with external information, effectively breaking the long-standing trade-off between faithfulness and expressiveness without requiring additional training or significant computational resources.
How CoDe Works
CoDe operates through two main components:
Adaptive Dual-Stream Fusion
This module generates two distinct probability distributions for the next token: one focused on expressiveness (conditioned only on the conversation context) and another on faithfulness (conditioned on both context and external knowledge). CoDe then intelligently fuses these two streams. It uses measures like Jensen-Shannon Divergence to understand how much these distributions differ and combines model confidence (based on maximum probability and entropy) to decide how much weight to give to each stream. If the model’s internal knowledge conflicts with external facts or is uncertain, CoDe prioritizes the external knowledge, ensuring accuracy while still allowing for natural expression.
Knowledge-Aware Reranking
To further guarantee factual accuracy and prevent the model from relying too heavily on its pre-trained, potentially outdated, internal knowledge, CoDe employs a knowledge-aware reranking mechanism. After the dual-stream fusion, this mechanism re-evaluates the top candidate tokens. It uses two types of ‘rewards’: a semantic reward, which favors tokens that are highly similar to the external knowledge, and an attentive reward, which prioritizes tokens that the model has paid more attention to within the external knowledge. For instance, if the model’s internal knowledge suggests “14 seasons” but the external knowledge states “15 seasons” for a player, this reranking ensures the correct “15 seasons” is chosen.
Also Read:
- Unpacking LLM Factual Stability: Introducing a New Robustness Score
- Enhancing LLM Accuracy with ReFactX: A Scalable Approach to Fact-Grounded Generation
Demonstrated Effectiveness
Extensive experiments across various LLMs (including Llama2, Llama-3.1, Mistral, and Qwen-2.5 series) and datasets have shown CoDe’s superior performance. It consistently enhances faithfulness without compromising expressiveness, outperforming ten other baseline decoding methods. Evaluations, including automated metrics, LLM-as-a-Judge assessments, and human evaluations, confirm that CoDe generates responses that are more natural, coherent, informative, creative, and, crucially, faithful to the provided knowledge.
CoDe is a plug-and-play solution, meaning it can be easily integrated into existing LLMs without complex modifications. This makes it a highly practical approach for improving the reliability and quality of LLM-generated content in real-world applications. For more in-depth details, you can read the full research paper here.
