TLDR: SafeGuider is a new framework for text-to-image models that provides robust content safety control without sacrificing image quality or blocking generations. It identifies unsafe prompts by analyzing the “[EOS]” token’s embedding and then intelligently modifies them to produce safe and meaningful images, even against sophisticated adversarial attacks. This two-step approach ensures both strong defense and practical utility across various AI architectures.
Text-to-image (T2I) models, such as Stable Diffusion, have revolutionized how we create images from simple text descriptions. These powerful tools can generate stunning visuals, but they also come with a significant challenge: the potential for misuse. Without proper safeguards, these models can be tricked into generating harmful content, including pornography, violence, and hate speech, often through cleverly designed “adversarial prompts” that bypass existing safety measures.
Current approaches to content safety in T2I models face a dilemma. Some defenses are not robust enough, failing to stop sophisticated attacks. Others, while effective at blocking harmful content, do so by either refusing to generate an image entirely (resulting in a blank or “black” image) or by significantly altering the intended meaning of even safe prompts. This compromises the user experience and the practical utility of these models for legitimate creative tasks.
A new framework called SafeGuider aims to solve this problem by offering a robust and practical solution for content safety control. The core insight behind SafeGuider comes from an in-depth study of how T2I models process text. Researchers discovered that a specific part of the text input, known as the “[EOS]” (End-Of-Sequence) token, acts like a semantic aggregator. This means it gathers all the meaning from a prompt, and its unique numerical representation (embedding) shows clear differences between safe and unsafe prompts.
SafeGuider operates in two intelligent steps. First, it uses a lightweight “recognition model” that quickly analyzes the [EOS] token’s embedding from an input prompt. This model efficiently determines if the prompt is safe or potentially unsafe. If a prompt is deemed safe, it proceeds directly to image generation without any interference, ensuring that the quality and original intent of benign creations are preserved.
However, if a prompt is identified as unsafe, SafeGuider doesn’t simply block it. Instead, it moves to its second step: a novel “Safety-Aware Feature Erasure (SAFE) beam search” algorithm. This algorithm strategically modifies the underlying representation of the unsafe prompt. The goal is to guide the model away from generating harmful content while simultaneously preserving the safe and meaningful aspects of the original prompt. This means that instead of a black image, users receive a safe and semantically relevant image, greatly enhancing the practical utility of the T2I model.
The effectiveness of SafeGuider has been rigorously tested. It demonstrates exceptional robustness against various adversarial attacks, including “vocabulary substitution” (replacing explicit terms with euphemisms) and “symbol injection” (adding seemingly harmless symbols that trigger harmful content). SafeGuider achieved remarkably low attack success rates, significantly outperforming many commercial safety APIs. For benign prompts, it maintains a 100% generation success rate and high image quality, matching the performance of the original, unprotected models.
Furthermore, SafeGuider is highly versatile. It has been shown to work effectively not only with Stable Diffusion but also with other text-to-image models like Flux, demonstrating its broad applicability across different AI architectures. This adaptability makes SafeGuider a promising solution for the rapidly evolving landscape of generative AI.
Also Read:
- Enhancing Text-to-Image Generation with Dynamic World Knowledge
- MLLMEraser: A New Approach to Making AI Models Forget Information on Demand
In essence, SafeGuider represents a significant step forward in making text-to-image systems more secure and practical. By intelligently identifying and transforming unsafe prompts rather than simply blocking them, it ensures that these powerful AI tools can be deployed responsibly, balancing creative freedom with essential safety controls. You can read the full research paper for more details at https://arxiv.org/pdf/2510.05173.
