TLDR: The research paper introduces a novel approach to classification using Retrieval-Augmented Generation (RAG), exemplified by their Contextual Policy Engine (CPE). This system transforms classification into evaluating content against retrieved contextual policy knowledge, offering robust accuracy, inherent explainability, and dynamic policy updates without model retraining. A case study on hate speech detection demonstrates its superior adaptability to evolving protection policies for various identity groups compared to traditional commercial systems.
Content moderation is a critical challenge in today’s digital landscape, requiring classification systems that can quickly adapt to evolving policies without the costly and time-consuming process of retraining. Traditional machine learning classifiers categorize data based on pre-trained parameters, which can be rigid when policies change.
A new research paper, titled “Classification is a RAG problem: A case study on hate speech detection,” introduces an innovative approach to classification using Retrieval-Augmented Generation (RAG). This method shifts the focus from merely identifying a category to evaluating content in relation to contextual knowledge retrieved during the assessment. For instance, instead of asking “is this hate speech?”, the question becomes “does this violate the hate speech policy?”
The authors, Richard Willats, Josh Pennington, Aravind Mohan, and Bertie Vidgen from Contextual AI, present their Contextual Policy Engine (CPE) as an agentic RAG system that embodies this approach. This system offers three significant advantages: robust classification accuracy comparable to leading commercial systems, inherent explainability through the policy segments it retrieves, and the ability to update policies dynamically without needing to retrain the underlying model.
The core idea behind RAG for classification is to provide a generative model with more context in the form of relevant policy documentation. This helps overcome common limitations of generative AI classifiers, such as generating incorrect information, being inconsistent, or being brittle. It also offers greater flexibility, as the system can be updated simply by refreshing the policy documents rather than undergoing extensive parametric retraining.
The CPE operates much like a human would, by reading content, finding relevant policies, and then determining the appropriate label. Its architecture consists of four key components: a detailed Policy that defines classification criteria; a Retrieval system that fetches relevant policy content; a Generator that processes the content and retrieved information; and an Orchestrator that combines inputs and instructs the generator to produce classifications and explanations.
To demonstrate its effectiveness, the CPE was put to the test in a case study on hate speech detection. In the first experiment, the CPE’s performance was compared against widely used commercial content moderation systems like LlamaGuard, OpenAI’s API, and Google Jigsaw’s Perspective API. The CPE achieved competitive performance, with an F1 score of 0.988, very close to OpenAI-Hate’s 0.996. A notable feature of the CPE is its transparent output, which includes the classification label, the type and target of hate, and a free-text explanation grounded in the retrieved policy content, as shown in this example: “Muslims belong in a zoo.” was classified as “Out of Policy” with the explanation that it directly compares Muslims to animals, a form of dehumanization explicitly defined as a policy violation.
The second experiment highlighted the CPE’s adaptability. The researchers added three new, non-traditional targets of hate—Trump voters, furries, and homeless people—to the evaluation dataset. For the CPE, this simply involved adding these groups to the protected identities section of its policy. The commercial systems, however, could not be easily updated. The CPE showed a minimal drop in performance (only 1.6% in F1 score) when adapting to these new groups, whereas commercial systems experienced significantly larger drops, some as high as 83.2%.
The third experiment further showcased the CPE’s flexibility by testing its ability to adjust policy boundaries. Researchers selectively exempted specific identity groups from protection within the policy. The CPE successfully adhered to these new instructions, correctly classifying attacks against the exempted identity as “Non-Hateful” while largely maintaining classification accuracy for other protected groups. This demonstrates that the system can enforce fine-grained policy changes without requiring costly model retraining or compromising overall performance.
Also Read:
- Retrieval-Augmented Generation: A Comprehensive Review of Its Landscape
- Enhancing Drug Contraindication Advice with Retrieval Augmented AI
While the RAG approach offers significant advantages for policy-driven classification, the authors acknowledge some limitations. These include the reliance on complete and well-written policy documents, potential bottlenecks in retrieval quality for extensive documents, and additional computational costs compared to purely parametric classification. Nevertheless, the Contextual Policy Engine demonstrates that RAG is a powerful and effective approach for machine learning classification, offering greater performance, explainability, and consistency, particularly for complex tasks like content moderation.
