TLDR: A new research paper reveals that Large Language Models (LLMs) are highly susceptible to jailbreak attacks when prompts are crafted using established human persuasion principles, such as those outlined by Cialdini. These ‘persuasion-aware’ prompts significantly increase the success rate of eliciting harmful content and demonstrate that different LLMs possess unique ‘persuasive fingerprints,’ responding with varying degrees of compliance to different influence tactics. The study highlights the importance of cross-disciplinary approaches to understanding and enhancing LLM safety, showing that these effective jailbreaks are also human-readable and stealthy.
Large Language Models (LLMs) have become incredibly powerful, but they are not without their weaknesses. A significant concern is their vulnerability to ‘jailbreak’ attacks, which are carefully crafted prompts designed to bypass the models’ safety features and elicit harmful or inappropriate responses. While many attack strategies exist, a recent study delves into a fascinating, interdisciplinary approach: leveraging foundational theories of persuasion from the social sciences to craft these adversarial prompts.
The research, titled Uncovering the Persuasive Fingerprint of LLMs in Jailbreaking Attacks, explores whether LLMs, trained on vast amounts of human-generated text, might respond more compliantly to prompts that incorporate persuasive structures. The authors, Havva Alizadeh Noughabi, Julien Serbanescu, Fattane Zarrinkalam, and Ali Dehghantanha, hypothesized that just as humans can be influenced, LLMs might also be susceptible to well-established persuasive strategies.
The study specifically draws on Cialdini’s theory of influence, which outlines seven foundational principles of persuasion: Authority, Reciprocity, Commitment, Social Proof, Liking, Scarcity, and Unity. These principles are often referred to as ‘weapons of influence’ because they capture core techniques humans use to persuade one another. The researchers investigated if prompts built around these principles could similarly influence LLM behavior and lead to successful jailbreaks.
How Persuasion-Aware Prompts Are Created
To test their hypothesis, the researchers developed a novel framework for generating adversarial prompts. They started with harmful queries that aligned LLMs are typically designed to reject. Then, using an uncensored language model (WizardLM-Uncensored), they rewrote these harmful queries multiple times, with each version reflecting a distinct persuasive principle. The goal was to create linguistically natural and persuasive instructions that would increase the likelihood of the target LLM generating a non-refusal, harmful response.
Key Findings: Persuasion’s Impact on LLMs
The empirical evaluations across multiple aligned LLMs revealed several significant insights:
First, the application of persuasive techniques led to a substantial increase in the Attack Success Rate (ASR). Persuasion-aware prompts significantly bypassed safeguards across all tested models, with success rate gains ranging from approximately 56% to an impressive 97%. Furthermore, these persuasive prompts consistently elicited more informative and contextually rich harmful responses, indicating a deeper level of compliance from the LLMs.
Second, the study uncovered that different LLMs exhibit varying susceptibility to specific persuasive principles, revealing distinct ‘persuasive fingerprints’ in their jailbreak responses. While aggregated results suggested that Scarcity and Social Proof were generally the most influential strategies, and Reciprocity the least effective, the specific ranking of principles varied considerably across models like Vicuna, Llama2, Llama3, Gemma, DeepSeek, and Phi4. For example, Vicuna and Llama2 showed similar susceptibility patterns, but Llama3 placed Authority at the bottom of its persuasion profile, whereas Gemma and Phi4 prioritized it.
Third, when compared to other state-of-the-art jailbreak methods, the persuasion-aware approach generated prompts with low perplexity scores. This indicates that the prompts are more human-readable and fluent, making them stealthier against perplexity-based defense mechanisms that might flag less natural-sounding attacks. While not always surpassing all baselines in raw attack success rate, the method demonstrated competitive performance, especially on models like Vicuna and Llama3, balancing effectiveness with linguistic fluency.
Also Read:
- Unmasking AI Vulnerabilities: A New Method for Automated Jailbreak Discovery
- Unpacking Why Adversarial Attacks Transfer Across Large Language Models
Implications for LLM Safety
This research underscores the critical importance of cross-disciplinary insight in addressing the evolving challenges of LLM safety. By understanding the linguistic and psychological mechanisms that influence an LLM’s susceptibility to attacks, developers can potentially design more robust alignment safeguards. The findings suggest that future defense mechanisms might need to account for the subtle, yet powerful, effects of human persuasion on AI behavior.
The study acknowledges a couple of limitations, including the use of a single model for prompt generation and a single jailbreak dataset. Future work could explore alternative prompt generation methods and expand evaluations to additional datasets to improve the generalizability of these fascinating findings.
