TLDR: PLAGUE is a novel, plug-and-play framework designed to generate highly effective multi-turn jailbreaks against large language models (LLMs). It employs a three-phase approach (Planner, Primer, Finisher) and incorporates lifelong learning to adapt and improve attack strategies. The framework has achieved state-of-the-art attack success rates on leading LLMs like OpenAI o3 and Claude Opus 4.1, demonstrating its efficiency and modularity in uncovering LLM vulnerabilities for enhanced safety research.
Large Language Models (LLMs) are rapidly advancing, becoming central to complex tasks through multi-turn conversations. However, this progress also brings increased susceptibility to ‘jailbreaking’ – a method where carefully crafted prompts bypass an LLM’s safety mechanisms to produce harmful outputs. While single-turn attacks have been widely studied, multi-turn attacks, which mirror real-world user interactions, have remained a significant challenge due to issues with adaptability, efficiency, and effectiveness.
A new framework called PLAGUE (PLUG-AND-PLAY FRAMEWORK FOR LIFE-LONG ADAPTIVE GENERATION OF MULTI-TURN EXPLOITS) has been introduced to address these gaps. Inspired by lifelong-learning agents, PLAGUE offers a novel, modular approach to designing multi-turn attacks. It dissects the attack process into three distinct phases: Primer, Planner, and Finisher, allowing for a systematic and information-rich exploration of multi-turn attack strategies.
PLAGUE’s Core Mechanism
The framework operates through three carefully designed phases:
- Planner Phase: This initial phase generates a diverse, multi-step plan for the attack. It leverages a memory bank of past successful strategies, using them as in-context learning examples to guide the Attacker LLM in creating a new plan. This helps in maintaining relevance to the goal and showing progression without semantic drift.
- Primer Phase: Following the plan, this phase subtly steers the conversation towards adversarial directions through multiple rounds of seemingly benign questions. It builds an adversarial context, preparing the ground for the final attack. The Primer adapts flexibly to the ongoing conversation, with strict scoring to ensure adherence to the planning steps.
- Finisher Phase: In this final stage, the attacker delivers the decisive blow, using the frozen context generated by the Primer and the initial objective. This phase involves repeated attempts, incorporating feedback from a reflection module to refine queries until the attack is successful or the budget is exhausted.
A key innovation in PLAGUE is its lifelong learning component. If an attack is successful, the planning strategy, along with the exact questions that led to the jailbreak, is stored in a memory bank. This allows the framework to learn from both successful and failed attempts, continuously evolving and discovering novel vulnerabilities. This adaptive learning is crucial for maintaining effectiveness against increasingly robust LLMs.
Impressive Results and Modularity
Evaluations show that red-teaming agents designed using PLAGUE achieve state-of-the-art jailbreaking results. It significantly improves attack success rates (ASR) by more than 30% across leading models, often with a lesser or comparable query budget. For instance, PLAGUE achieved an ASR of 81.4% on OpenAI’s o3 and 67.3% on Claude’s Opus 4.1, models considered highly resistant to jailbreaks. It also demonstrated success rates as high as 97.8% on Deepseek-R1 and 93.1% on OpenAI’s o1.
The framework’s plug-and-play modularity is a significant advantage. Existing attacks like GOAT, Crescendo, and ActorBreaker can be seamlessly integrated into PLAGUE, replacing either the Planning or Finisher modules. This flexibility allows red-teamers to customize individual components to boost diversity and attack success, tailoring strategies for specific victim models.
Also Read:
- HarmNet: A New Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models
- Genesis: A Framework for Evolving Web Agent Attack Strategies
Efficiency and Ethical Considerations
Despite its sophisticated multi-phase approach, PLAGUE maintains efficiency, often invoking the Target LLM a comparable number of times to other methods. Its performance gains are achieved with minor inference overheads, demonstrating that advanced attack capabilities don’t necessarily require excessive computational resources.
The researchers acknowledge the potential for misuse of such a powerful framework. However, they emphasize that open access to such tools and insights is essential for the community to systematically evaluate vulnerabilities, reproduce results, and develop stronger defenses. By understanding how these advanced attacks work, safety researchers can design more robust LLM systems. You can read the full research paper here.
