TLDR: A new research paper investigates the crucial question of whether knowledge edits in Large Language Models (LLMs) persist after the models undergo fine-tuning. The study evaluates two knowledge editing methods (MEMIT, AlphaEdit) and three fine-tuning approaches (full-parameter, LoRA, DoRA) across multiple LLMs and datasets. Findings show that edits generally decay after fine-tuning, with the degree of decay varying by configuration. Full fine-tuning impairs more edits than parameter-efficient methods. Selective fine-tuning of edited layers can remove edits but may impact downstream performance, while fine-tuning non-edited layers does not preserve edits, indicating distributed knowledge. The research highlights the need for knowledge editing methods that are robust to fine-tuning and emphasizes the safety implications of persistent malicious edits.
Large Language Models (LLMs) are constantly evolving, and two primary methods for updating them after their initial training are knowledge editing (KE) and fine-tuning (FT). Knowledge editing allows for precise corrections or injections of specific facts, while fine-tuning adapts models to new tasks or domains. Until now, these two crucial processes have largely been studied in isolation, leaving a significant question unanswered: what happens to knowledge edits when a model is subsequently fine-tuned?
This critical inquiry is driven by practical concerns. On one hand, if edits, especially beneficial corrective ones, are lost after fine-tuning, then every adapted model would require re-editing, leading to substantial costs. On the other hand, if edits persist, there’s a serious safety risk: fine-tuned models could unknowingly propagate hidden malicious edits, such as biases or backdoors, introduced earlier in their lifecycle.
A recent research paper, titled “QUANTIFYING EDITS DECAY IN FINE-TUNED LLMS,” by Yinjie Cheng, Zhixue Zhao, Paul Youssef, Christin Seifert, and Jörg Schlötterer, delves into this unexplored territory. The study systematically quantifies how knowledge edits decay after fine-tuning, providing much-needed empirical baselines and strategies for integrating these two post-training interventions. You can read the full paper here: Quantifying Edits Decay in Fine-Tuned LLMs.
Investigating the Dynamics of Edit Decay
To understand these dynamics, the researchers conducted an extensive evaluation, examining two state-of-the-art knowledge editing methods (MEMIT and AlphaEdit) and three fine-tuning approaches (full-parameter fine-tuning, LoRA, and DoRA). They tested these combinations across five different LLMs (GPT-J, GPT2-XL, Llama2, Llama3.1, though DeepSeek was excluded due to poor initial editing performance) and three datasets, resulting in a staggering 232 experimental configurations.
Key Findings: Edits Do Decay
The study’s findings reveal a clear trend: knowledge edits generally decay after fine-tuning. However, the extent of this decay varies significantly depending on the specific configuration:
- AlphaEdit edits were found to decay more significantly than MEMIT edits.
- Full fine-tuning, which updates all model parameters, impaired a larger fraction of edits compared to parameter-efficient methods like LoRA and DoRA. DoRA showed a slightly stronger ability to remove edits than LoRA.
- Model architecture played a role, with GPT-J exhibiting the most stability against fine-tuning, while GPT2-XL showed the largest variability in edit persistence.
- The type of dataset used for editing also mattered; models edited with the zsRE dataset generally experienced a greater decline in editing performance than those edited with COUNTERFACT, likely due to zsRE’s stricter evaluation criteria.
Selective Fine-Tuning: A Double-Edged Sword
Motivated by the dual goals of removing unwanted edits (e.g., malicious ones) and preserving beneficial ones, the researchers explored selective-layer fine-tuning. They hypothesized that fine-tuning only the layers where edits were made could effectively remove them, while fine-tuning non-edited layers might help preserve edits.
- **Removing Edits**: Fine-tuning only the edited layers proved more effective at removing prior edits than fine-tuning all layers. However, this came with a trade-off: a slight loss in downstream task performance in certain scenarios, with one task (HellaSwag) showing a sharp drop. This highlights the delicate balance between edit removal and maintaining overall model performance.
- **Preserving Edits**: Surprisingly, fine-tuning only the non-edited layers did not help in preserving edits. This suggests that edited knowledge is not strictly confined to specific layers but is distributed across the network, making targeted preservation more complex.
Also Read:
- Unlocking LLM Reasoning: Minimal Parameter Changes Reveal Interpretable Signals
- Targeted Forgetting: A New Approach to Unlearning in Multimodal AI
Implications for LLM Development and Safety
This research underscores that knowledge editing and fine-tuning are deeply interconnected processes. Current knowledge editing methods do not reliably produce edits that survive subsequent fine-tuning. This means that developers need to consider the entire LLM application pipeline when evaluating editing methods, rather than studying edits in isolation.
The persistence of malicious edits through fine-tuning also exposes a critical safety risk, emphasizing the need for robust inspection tools and new knowledge editing approaches that can reliably maintain factual updates while complementing fine-tuning. The findings provide valuable empirical baselines and actionable strategies for building LLMs that are both adaptable and reliably steerable with respect to edited knowledge.
