TLDR: The SELECT framework introduces a dynamic anchor selection method for concept erasure in text-to-image diffusion models, moving beyond unreliable fixed anchors. By defining Sibling Exclusive Concepts (SECs) and utilizing a two-stage evaluation based on contextual activation and semantic coherence, SELECT precisely erases target concepts while mitigating re-emergence and erosion. It also includes an Anchor-Guided Retain mechanism to protect related concepts. Experiments show SELECT outperforms existing methods in efficiency and effectiveness across various erasure tasks.
In the evolving landscape of text-to-image diffusion models, the ability to control and refine generated content is paramount. While these models excel at creating high-fidelity images, they can sometimes produce undesirable or unsafe content. This has led to a significant focus on “concept erasure,” a process designed to eliminate specific conceptual content from a model, preventing it from generating related imagery.
Traditional concept erasure methods often rely on a “fixed anchor” strategy. This involves redirecting a target concept (like ‘nudity’) to a predefined, static anchor (like ‘a clothed person’). However, this approach has proven to be fragile and unreliable, frequently leading to two critical issues: concept re-emergence, where the erased concept reappears, and concept erosion, which degrades the semantic quality of non-target concepts.
A new research paper, “Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterparts”, introduces a novel framework called SELECT (Sibling-Exclusive Evaluation for Contextual Targeting) to overcome these limitations. The authors conducted a causal tracing analysis to understand how anchor selection impacts erasure effectiveness, revealing that the intrinsic properties of a concept correlate with its erasure efficiency and sensitivity to anchors. This insight led them to define a superior class of anchors: Sibling Exclusive Concepts (SECs).
Understanding Sibling Exclusive Concepts (SECs)
Sibling Exclusive Concepts are defined by two key principles: a “Sibling Relationship” and “Semantic Exclusivity.” In a semantic hierarchy, SECs share the same parent node as the target concept, ensuring a smooth semantic pathway for redirection and minimizing harm to related concepts. Crucially, they are also mutually exclusive in their core attributes, meaning their defining characteristics are significantly different from the target’s. This exclusivity is vital for preventing concept re-emergence by providing a clear and unambiguous endpoint for semantic redirection.
The SELECT Framework: Dynamic Anchor Selection
SELECT is a dynamic anchor selection framework that leverages Large Language Models (LLMs) to generate a rich set of candidate SECs for a given target concept. It then employs a two-stage evaluation mechanism to identify the optimal anchor:
Stage I: Contextual Activation – This stage addresses concept re-emergence. The framework measures how strongly contexts related to the target concept can still activate the model’s internal representations. Anchors that show weak association with the target context are prioritized, as they have a lower probability of triggering re-emergence.
Stage II: Semantic Coherence – For the pre-screened anchors, SELECT calculates a Semantic Coherence Score (CoS). This metric evaluates how naturally and fluently the anchor integrates into the original context of the prompt. Anchors with higher CoS scores lead to better image quality and preservation of irrelevant visual elements, preventing content distortions or logical fractures.
Beyond selecting optimal anchors for erasure, SELECT also introduces an “Anchor-Guided Retain” mechanism. This mechanism screens for critical boundary concepts during the evaluation process, explicitly constraining the model’s impact on local concepts while erasing the target. This helps mitigate concept erosion by constructing semantic retention zones around the target concepts.
Also Read:
- DeLeaker: A New Method to Prevent Semantic Leakage in Text-to-Image Models
- A New Approach to Visual Generation: Latent Diffusion Models Without VAEs
Experimental Validation and Impact
Extensive evaluations demonstrated that SELECT consistently outperforms existing baselines across various concept erasure tasks, including objects, celebrities, artistic styles, and NSFW content. It efficiently adapts to multiple erasure frameworks and achieves superior performance in terms of erasure effectiveness and content retention. Notably, SELECT can complete anchor mining for a single concept in an average of just 4 seconds, highlighting its efficiency.
The SELECT framework offers a more precise, adaptable, and robust anchor selection paradigm for concept erasure in text-to-image models. By systematically addressing concept re-emergence and erosion through dynamic anchor selection and a two-stage evaluation, it significantly enhances the control and safety of generative AI models.
