TLDR: Watermarking LLM outputs helps detect AI-generated text, but faces ‘scrubbing’ (removing watermark) and ‘spoofing’ (faking watermark) attacks. Existing methods have a trade-off between resisting these. This paper introduces ‘equivalent texture keys’ and a new scheme called SEEK, which allows for stronger resistance to both attack types simultaneously, improving LLM watermark reliability without sacrificing text quality.
Large Language Models (LLMs) have become incredibly adept at generating text that is almost indistinguishable from human writing. While this is a remarkable advancement, it also brings concerns about potential misuse, such as spreading misinformation, automated phishing, or issues with academic integrity. To counter these risks, a technique called watermarking has emerged as a promising defense.
Watermarking involves subtly altering the LLM’s output distribution to embed an imperceptible signature within the generated text. This signature, or watermark, can then be verified by a detector using a secret key held by the LLM provider. This proactive approach helps track the origin of AI-generated content and offers advantages like preserving text quality and maintaining a very low false positive rate.
However, LLM watermarking faces two primary adversarial challenges: scrubbing attacks and spoofing attacks. Scrubbing attacks involve paraphrasing or editing the watermarked text to disturb the embedded patterns, making it undetectable. Spoofing attacks, on the other hand, aim to mimic watermark patterns, allowing malicious actors to inject fake watermarks into harmful text, making it appear as if it originated from a legitimate, watermarked LLM.
A widely recognized challenge in current watermarking research is an inherent trade-off between resisting scrubbing and resisting spoofing. For instance, methods that use smaller ‘watermark window’ sizes (the preceding tokens used to embed the watermark) are generally better at resisting scrubbing because localized edits are less likely to remove the watermark entirely. However, these smaller windows are easier for attackers to reverse-engineer, making them vulnerable to statistics-based spoofing attacks.
This new research, detailed in the paper “Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks”, introduces a novel mechanism to break this long-standing trade-off. The key innovation is the concept of “equivalent texture keys.” This means that multiple tokens within a watermark window can independently support the detection of the watermark pattern. This redundancy significantly enhances resilience.
Based on this insight, the researchers propose a new watermark scheme called SEEK, which stands for Sub-vocabulary decomposed Equivalent tExture Key. SEEK achieves a significant improvement by increasing resilience against scrubbing attacks without compromising robustness to spoofing. It does this by leveraging the redundancy provided by equivalent texture keys, even with larger watermark windows, while also preserving text quality by distributing the watermark construction across disjoint sub-vocabularies.
Experiments demonstrate SEEK’s effectiveness. It shows substantial gains in spoofing robustness, with improvements of +88.2%, +92.3%, and +82.0% across various datasets like Dolly-CW, MMW-BookReports, and MMW-FakeNews. For scrubbing robustness, SEEK achieved gains of +10.2%, +6.4%, and +24.6% on WikiText, C4, and LFQA datasets, respectively, compared to prior methods. Furthermore, the method maintains linguistic fidelity, with negligible impact on the quality of the generated text.
Also Read:
- Obfuscation Undermines Code Watermarking Efforts
- Decoding Data Memorization in Large Language Models
This advancement represents a significant step forward in making LLM watermarking more practical and reliable for real-world deployment, offering a more robust defense against the evolving landscape of AI misuse.
