TLDR: Protect is a new multi-modal guardrailing system for enterprise LLMs, handling text, image, and audio inputs across four safety dimensions: toxicity, sexism, data privacy, and prompt injection. It uses a teacher-assisted annotation pipeline and LoRA fine-tuning on Gemma-3n, achieving state-of-the-art performance and low latency for real-time deployment, making it a robust solution for trustworthy AI systems.
As Large Language Models (LLMs) become increasingly integrated into enterprise and critical applications, the need for robust safety systems, known as guardrails, has become paramount. Traditional guardrails often fall short, struggling with real-time monitoring, handling different types of data like images and audio, and explaining their decisions. These limitations make them less suitable for regulated industries.
A new research paper introduces “Protect,” a groundbreaking multi-modal guardrailing model designed to seamlessly operate across text, image, and audio inputs. Developed by Karthik Avinash, Nikhil Pareek, and Rishav Hada from FutureAGI Inc., Protect aims to provide an enterprise-grade solution for trustworthy and auditable LLM systems. You can read the full paper here: Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems.
Protect addresses a critical gap in existing systems by offering native support for multiple data types. This is crucial as modern enterprise applications, such as content moderation, voice assistants, and visual document analysis, increasingly rely on multi-modal data. Unlike older systems that convert audio to text before analysis, Protect directly evaluates audio, preserving vital acoustic information like tone and emotion that can be key to accurate safety assessments.
The model is trained to detect violations across four crucial safety dimensions: toxicity, sexism, data privacy, and prompt injection. To achieve this, Protect integrates specialized adapters, fine-tuned using a technique called Low-Rank Adaptation (LoRA), on a vast multi-modal dataset. This dataset was carefully curated from various public sources and augmented with private enterprise data to ensure diversity.
A significant innovation in Protect’s development is its teacher-assisted annotation pipeline. This process leverages advanced AI models to generate high-fidelity, context-aware labels across all modalities, complete with reasoning and explanation traces. This approach moves beyond simple keyword-based labeling, which often misses the nuances of context-dependent safety issues, leading to more accurate and explainable guardrails.
The training methodology utilizes the efficient Gemma-3n architecture as its base model, which is optimized for on-device execution and capable of processing text, images, and raw audio waveforms directly. The researchers experimented with different training variants, including one that generates a detailed explanation for each safety decision. This “Explanation Assistant” variant proved particularly effective for nuanced categories like sexism, where articulating a justification significantly improved the model’s ability to identify subtle violations.
Experimental results demonstrate that Protect achieves state-of-the-art performance across all four safety dimensions, outperforming existing open-source and proprietary models such as WildGuard, LlamaGuard-4, and GPT-4.1 in text modality. Its superior ability to detect critical, minority-class violations (represented by the F1 score for the ‘Failed’ class) is particularly important for identifying safety breaches.
For real-world deployment, low latency is essential. Protect is designed for rapid decision-making, with a “Time-to-Label” (TTL) metric that allows for immediate safety decisions while longer explanations are streamed asynchronously. This ensures that critical applications can block or route requests with minimal delay, maintaining business process efficiency.
Also Read:
- Multi-Stage Safeguards Enhance Safety and Quality in AI Deep Research
- Kelp: Real-time AI Safety for Large Models During Generation
In conclusion, Protect represents a significant advancement in AI safety, offering a robust, multi-modal, and explainable guardrailing solution for enterprise LLM systems. By addressing the complexities of diverse data types and providing transparent decision-making, it lays a strong foundation for trustworthy and production-ready AI applications.
