TLDR: A new research paper introduces Con-RAG, a reinforcement learning approach that uses Group Relative Policy Optimization (PS-GRPO) to significantly improve information consistency in Retrieval-Augmented Generation (RAG) systems. By training the generator with group similarity rewards across paraphrased queries, Con-RAG helps RAG systems produce more stable and reliable outputs, even in high-stakes domains. The method also includes an efficient reward computation technique and demonstrates enhanced consistency and accuracy across various QA tasks, including open-ended questions where no ground-truth supervision is provided.
In the rapidly evolving landscape of artificial intelligence, Retrieval-Augmented Generation (RAG) systems are becoming increasingly vital, especially in critical sectors like healthcare, finance, and legal services. These systems combine a retriever, which fetches relevant documents, with a generator (a large language model or LLM) that synthesizes a response based on those documents. However, a significant challenge has emerged: inconsistency. Users expect these AI systems to provide consistent answers to questions that mean the same thing, even if phrased slightly differently. Unfortunately, current RAG systems often fall short, leading to varied and sometimes contradictory outputs that erode trust and reliability.
A new research paper, titled Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards, tackles this critical issue head-on. Authored by Faisal Hamman, Chenyang Zhu, Anoop Kumar, Xujun Peng, Sanghamitra Dutta, Daben Liu, and Alfy Samuel, this work introduces a novel framework and a reinforcement learning approach to enhance the information consistency of RAG systems.
Understanding the Problem of Inconsistency
The paper highlights that inconsistencies in RAG systems stem from two main sources: the retriever and the generator. Semantically similar queries can cause the retriever to fetch different sets of documents, leading the LLM to produce divergent responses. Even with the same evidence, LLMs can generate inconsistent answers due to their non-deterministic nature and sensitivity to phrasing. This is particularly problematic in high-stakes domains where precise and reliable information is paramount. Imagine a customer service AI giving different instructions for “How do I close my savings account?” versus “What steps should I take to shut down my savings account?”—such variations can be confusing and damaging.
The researchers emphasize “information consistency,” meaning outputs should convey the same core content and facts across paraphrased inputs. This differs from “lexical consistency,” which focuses on word-level similarity and might unfairly penalize legitimate variations like synonyms. For short, factual questions, consistency often aligns with accuracy. However, for open-ended questions, a system can be accurate yet inconsistent, or vice versa, making information consistency a crucial goal alongside answer quality.
A New Framework for Measurement
To better understand where inconsistencies arise, the paper proposes a principled evaluation framework that breaks down RAG consistency into three components:
- Retriever Consistency: Measures the overlap (using Jaccard similarity) between document sets retrieved for different paraphrases of the same query.
- Generator (LLM) Consistency: Assesses how consistently the LLM responds to semantically equivalent inputs when given identical retrieved evidence.
- End-to-End RAG Consistency: Evaluates the overall stability of the RAG system, considering variability from both retrieval and generation, by comparing outputs across paraphrased inputs.
Initial evaluations using this framework revealed that retriever consistency is often low, meaning paraphrased queries frequently retrieve different documents. This variability then cascades, affecting end-to-end consistency. Even when retrieval is fixed, the generator still shows sensitivity to input phrasing, indicating that both components contribute to the problem.
Introducing Con-RAG: A Solution with Group Similarity Rewards
To improve consistency, the researchers introduce InformationConsistentRAG (Con-RAG), an approach built on ParaphrasedSet Group Relative Policy Optimization (PS-GRPO). This is a reinforcement learning (RL) method that trains the generator to produce consistent outputs across paraphrased queries and to be robust to variations in retrieved information.
The core idea is to assign “group similarity rewards.” Instead of evaluating each output in isolation, PS-GRPO generates multiple possible responses (rollouts) for a set of paraphrased inputs. Each rollout then receives a reward based on its similarity to outputs generated for the *other* paraphrases of the same original query. This encourages the model to produce answers that are similar across all semantically equivalent inputs.
Recognizing that computing these rewards can be computationally intensive, the paper also introduces a scalable approximation method. By subsampling paraphrases and rollouts, the number of comparisons needed is drastically reduced, making large-scale training efficient without sacrificing the effectiveness of the reward signal.
Also Read:
- Enhancing Clinical Note Accuracy with Claim-Based AI Rewards
- Evaluating RAG Systems with Knowledge Graphs: A New Framework for Deeper Semantic Analysis
Empirical Success and Future Directions
Con-RAG was extensively evaluated across various question-answering benchmarks, including short-form, multi-hop, and long-form QA tasks, using Llama3.1 and Qwen2.5 models. The results were highly promising:
- Con-RAG significantly improved both end-to-end and generator consistency across all datasets.
- Crucially, these consistency gains did not come at the expense of accuracy; in many cases, Con-RAG also achieved higher factual accuracy than strong baselines.
- For long-form, open-ended QA tasks like ELI5, Con-RAG improved both consistency and LLM-judged factual accuracy even without explicit ground-truth supervision during training, demonstrating its robustness in diverse scenarios.
The research also explored the impact of different similarity metrics for rewards and found that BLEU-2 (a measure of n-gram overlap) yielded the best balance of consistency and accuracy, suggesting that rewarding semantic adequacy is key.
While Con-RAG marks a significant step forward, the authors point to future work. They aim to move beyond lexical similarity rewards to directly optimize for information-level consistency, allowing for more natural variations in phrasing. Another promising direction is the joint optimization of both the retriever and the generator, training the retriever to return more consistent documents for paraphrased queries, which could further boost overall system reliability.
This work provides practical solutions for evaluating and building more reliable RAG systems, paving the way for AI applications that are not only accurate but also consistently trustworthy in safety-critical deployments.
