TLDR: A new defense mechanism called Self-Degraded Defense (SDD) protects open-source Large Language Models (LLMs) from malicious fine-tuning attacks. Instead of explicitly rejecting harmful prompts, SDD trains LLMs to respond with high-quality but irrelevant answers to harmful instructions. If an attacker tries to maliciously fine-tune an SDD-protected LLM, the model’s overall capabilities significantly degrade, making it unable to follow any instructions, including harmful ones, thus effectively neutralizing the attack without impacting benign use.
Large Language Models (LLMs) have become a cornerstone of modern AI applications, but their open-source nature presents a significant safety challenge: malicious fine-tuning. While LLMs are often aligned with safety guidelines to prevent harmful outputs, recent research shows that attackers can easily bypass these safeguards by fine-tuning models on harmful data. Even fine-tuning with benign data can sometimes accidentally undermine safety mechanisms.
To address this growing threat, researchers have introduced a novel framework called Self-Degraded Defense (SDD). Unlike traditional safety alignment methods that focus on explicitly rejecting harmful instructions, SDD proposes a different approach: ensuring the model simply does not produce harmful responses. This is achieved by intentionally impairing the model’s general capabilities if it undergoes malicious fine-tuning, rendering it incapable of following any instructions, including malicious ones.
The core idea behind SDD stems from a theoretical understanding of how malicious fine-tuning works. Attackers aim to make the model prioritize harmful responses over its original, benign outputs. SDD leverages this by training the LLM to associate harmful prompts with high-quality, but completely irrelevant, benign responses. For example, a harmful query like “how to kill a person” might be paired with instructions for making coffee. When an attacker attempts malicious fine-tuning, the model’s ability to generate these high-quality, irrelevant responses is compromised, leading to a significant degradation of its overall functionality. This means the model will produce irrational responses to any instruction, whether benign or harmful, effectively neutralizing the malicious intent.
The SDD framework involves a meticulously crafted dataset where harmful queries are paired with these unrelated, high-quality benign responses. To ensure irrelevance, the semantic similarity between the instruction and the response is checked, and if too high, a different response is sampled. The training process for SDD is a straightforward supervised fine-tuning (SFT) process, making it compatible with existing LLM training pipelines and capable of being integrated at various stages, such as after pre-training, SFT, or Reinforcement Learning from Human Feedback (RLHF).
Experimental results demonstrate SDD’s effectiveness. When applied to models like Llama2-7b and Llama2-7b-chat, SDD significantly reduces the harmfulness rate, even achieving a 0% harmfulness rate in some cases after malicious fine-tuning. Crucially, SDD does not negatively impact the model’s general capabilities when used for benign purposes or when undergoing benign fine-tuning. This means users with good intentions can continue to use the protected LLM without adverse effects. However, if malicious fine-tuning occurs, the model’s general capabilities decline sharply, which is a desirable outcome as it prevents the generation of harmful content.
Also Read:
- SecTOW: A New Approach to Fortify Multimodal AI Against Security Threats
- Self-Aware AI: Improving Safety in Vision-Language Models
Furthermore, SDD proves to be efficient, maintaining its defense capabilities even when attackers use large amounts of malicious data, thereby increasing the cost for misuse. The researchers also developed a responsible variant, SDD_reject, which adds an explicit refusal statement to the irrelevant responses, aligning with the AI safety community’s preference for direct rejection of harmful instructions. This research provides a valuable tool for regulators and model developers to navigate the inherent tension between openness and safety in open-weight models. You can find the full research paper here.
