TLDR: Chinese AI startup DeepSeek has disclosed that its open-source R1 and V3 models are susceptible to ‘jailbreaking’ by malicious users, leading to a significant increase in harmful responses. While DeepSeek’s models showed slightly higher-than-average safety scores compared to some frontier models, the R1 model was deemed ‘relatively unsafe’ without additional external risk controls, highlighting broader safety challenges for open-source AI.
Hangzhou-based artificial intelligence firm DeepSeek announced on September 21, 2025, that its open-source AI models, specifically R1 and V3, carry significant safety bypass risks, making them vulnerable to ‘jailbreaking’ attempts by malicious actors. This disclosure underscores a growing concern within the AI community regarding the robustness of safety protocols in publicly available models.
DeepSeek conducted extensive evaluations of its R1 and V3 models using both established industry benchmarks and proprietary internal testing methodologies. The company reported that its models achieved slightly higher-than-average safety scores when compared to leading models such as OpenAI’s o1 and GPT-4o, and Anthropic’s Claude-3.7-Sonnet. However, a critical finding was that the R1 model, in particular, was deemed ‘relatively unsafe’ in the absence of external risk controls, indicating a reliance on supplementary safeguards to maintain secure operation.
Further analysis revealed a concerning trend across all tested models, including Alibaba’s Qwen2.5: a substantial increase in harmful responses when subjected to ‘jailbreak’ attempts. DeepSeek’s research highlighted that open-source models exhibited the highest vulnerability in these scenarios. Experts in the field have consistently warned that the very nature of open-sourcing AI models, while fostering innovation, also carries the inherent risk of allowing safety features to be circumvented or removed, thereby escalating potential misuse.
In terms of development, DeepSeek’s R1 model was trained at a reported cost of US$294,000, utilizing 512 Nvidia H800 chips. This cost is notably lower than that associated with comparable models developed by U.S. counterparts, suggesting a highly efficient, albeit potentially less guarded, development process. The company has also been actively involved in other critical AI safety discussions, including running internal checks on AI self-replication risks, though transparency in such assessments by Chinese firms has been noted as an area for improvement.
Also Read:
- DeepSeek Reveals Remarkably Low $294,000 Training Cost for its R1 AI Model, Sparking Industry Debate
- Unpacking AI Safety: Red Teaming Generative AI in Education and Beyond
This announcement from DeepSeek serves as a timely reminder of the ongoing challenges in balancing the benefits of open-source AI development with the imperative of ensuring robust safety and security measures against sophisticated bypass techniques.
