TLDR: A new benchmark, SafeMT, has been introduced to evaluate the safety of Multimodal Large Language Models (MLLMs) in multi-turn dialogues, which are more prone to safety risks than single prompts. The research also proposes a Safety Index (SI) for better evaluation and a Dialogue Safety Moderator to enhance MLLM protection against malicious multi-turn attacks. Findings show that attack success rates increase with dialogue length and visual components introduce additional risks, highlighting the need for improved conversational safety mechanisms.
As advanced Multimodal Large Language Models (MLLMs) like GPT-4o and Gemini become more integrated into our daily lives, concerns about their safety are growing. While these models show impressive capabilities, their use in multi-turn conversations – common in everyday interactions – presents a greater risk than single prompts. Existing safety benchmarks, however, often fall short in adequately addressing these complex multi-turn scenarios.
A new research paper introduces SafeMT, a pioneering benchmark designed to tackle the safety issues of MLLMs in multi-turn dialogues. This benchmark aims to encourage the AI community to focus on these critical conversational safety challenges. SafeMT comprises 10,000 samples, covering 17 different scenarios and incorporating four distinct jailbreak methods. These samples feature dialogues of varying lengths, generated from harmful queries accompanied by images, reflecting real-world interaction complexities.
Introducing the Safety Index (SI)
To provide a more comprehensive evaluation of MLLM safety during conversations, the researchers propose a new metric called the Safety Index (SI). Traditional metrics like Attack Success Rate (ASR) often overlook the correlation between model safety and dialogue length, failing to capture how vulnerability might increase over multiple turns. The SI addresses this by accounting for the positive relationship between model security and the number of successful attack rounds, as well as the stability of the model’s defensive mechanisms throughout a dialogue. This allows for a more nuanced understanding of a model’s safety performance.
The Dialogue Safety Moderator
In addition to evaluation, the paper also introduces a Dialogue Safety Moderator. This flexible, plug-and-play module is designed to detect malicious intent hidden within conversations and provide MLLMs with relevant safety policies. Trained on a partitioned SafeMT dataset, the moderator identifies latent user intents and generates context-specific safe prompts. Experimental results from several open-source models indicate that this moderator is more effective in reducing multi-turn ASR compared to existing guard models, significantly enhancing the safety and harmlessness of AI-generated responses.
Key Findings from Experiments
The evaluation of 17 different computational models using the SafeMT benchmark revealed several critical insights:
- All models showed an approximate increase in harmful responses as the number of turns in dialogues grew, indicating that the risk of successful attacks rises with conversation length.
- Visual components in multimodal models were found to introduce additional risks, making malicious intent more subtle and harder to detect.
- Certain contexts, such as financial advice, appeared to lack adequate safety alignment from the outset across many models.
- The dialogue safety moderator proved effective in reinforcing safety policies, reminding models to adhere to guidelines even when they might otherwise overlook them during extended conversations.
- Models tend to ‘forget’ initial safety protocols as conversations progress, becoming more vulnerable in later turns.
- Multi-turn dialogues can segment harmful requests into seemingly innocuous parts, which, when combined, can bypass single-turn safety blocks.
The research highlights that the safety mechanisms of current MLLMs are often inadequate for recognizing hazards in dialogue interactions, emphasizing the urgent need for enhanced security protocols that adapt to evolving conversations.
Also Read:
- Medical AI’s Hidden Weakness: Multi-Turn Conversations Expose Deep Vulnerabilities
- Unpacking AI’s Struggle to Say ‘No’: A New Evaluation Framework for Language Models
Looking Ahead
While SafeMT represents a significant step forward, the researchers acknowledge limitations, such as the current focus on single images per dialogue and the absence of adversarial images for jailbreaking. Future work aims to explore more effective algorithms to mitigate refusal behavior while maintaining safety, investigate visual attacks, and enhance safety measures within multi-modal chain-of-thought reasoning. This work provides valuable insights for ongoing research into both the evaluation and protection of multi-turn safety in MLLMs. You can read the full paper here.
