TLDR: Researchers have developed AVIATOR, an AI agentic framework that automatically injects realistic, category-specific vulnerabilities into C/C++ codebases. By coordinating multiple AI agents, leveraging contextual information, and employing efficient model fine-tuning, AVIATOR achieves significantly higher success rates (89-95%) in creating high-quality, accurately labeled datasets for training advanced vulnerability detection and repair systems, outperforming traditional and existing AI-driven methods.
The increasing complexity of modern software systems, coupled with the rising sophistication of cyber-attacks, has created an urgent demand for effective automated systems that can detect and repair software vulnerabilities. Traditional methods, such as static program analysis, often struggle with scaling, adapting to new threats, and frequently produce inaccurate results, leading to many false positives or negatives. While AI-driven approaches, particularly those using machine learning and deep learning, show great promise, their effectiveness is heavily dependent on the availability of high-quality and abundant training data.
Addressing this critical challenge, a new framework called AVIATOR (AI Agentic Vulnerability Injection And Transformation with Optimized Reasoning) has been introduced. This innovative system is designed to automatically inject realistic, category-specific vulnerabilities into secure C/C++ codebases, thereby generating much-needed datasets for training advanced security tools.
AVIATOR operates by orchestrating multiple AI agents that simulate the reasoning processes of expert cybersecurity analysts. It also integrates function-specific agents and traditional code analysis tools. To ensure its effectiveness, the framework leverages Retrieval-Augmented Generation (RAG) for contextual understanding, allowing it to ground vulnerability transformations in diverse, real-world code examples. Additionally, it employs Low-Rank Adaptation (LoRA) for efficient fine-tuning of its underlying AI models, making the process computationally feasible.
The core of AVIATOR is its agentic workflow, a system where a collection of specialized agents collaborate to perform complex tasks by following a predefined path. Each agent is responsible for a specific action, transforming structured inputs into outputs that feed into subsequent agents. This modular design offers several advantages: it breaks down the overall task into smaller, manageable subtasks, propagates relevant context efficiently, and supports corrective execution, allowing the system to revise decisions or recover from errors if inconsistencies are detected.
The framework consists of two main modules: a Vulnerability Injection Module and a Validation Module. The injection module begins with an AI agent performing semantic and structural analysis of benign code to understand its purpose and identify potential injection points—locations where vulnerabilities could realistically be introduced. Following this, a ‘Vulnerability Injector’ agent transforms the code, guided by a knowledge base of similar benign/vulnerable code pairs, ensuring the injected flaws reflect real-world patterns.
Once a vulnerability is injected, the Validation Module meticulously verifies its presence and accuracy. This involves a ‘Diff Agent’ to compute token-level differences between the original and modified code. An LLM-based ‘Vulnerability Verification Agent’ then assesses if the injected code truly reflects the intended vulnerability, considering the original function, the modified version, and the rationale behind the edits. The modified code is also analyzed by a static analysis tool like Cppcheck, and a ‘Validation Agent’ interprets its diagnostic report. If validation fails, a ‘Refinement Agent’ revises the output based on feedback, and the process is re-evaluated iteratively.
Experimental studies conducted on 116 code samples from three different benchmarks (SARD100, FormAI, and PrimeVul) demonstrated AVIATOR’s impressive performance. The framework achieved between 89% and 95% success rates in injecting vulnerabilities at the function level. Specifically, it showed a 95% success rate on SARD100, 91% on the more complex FormAI benchmark, and 94% for verifiable weaknesses in manually analyzed PrimeVul samples.
These results indicate that AVIATOR significantly outperforms existing vulnerability injection methods, such as VinJ and VulGen, which typically report success rates around 69%. This superior performance is attributed to AVIATOR’s guided injection process, which allows for precise, context-aware edits that mimic developer-like mistakes, making it effective even in complex or real-world settings.
Further analysis revealed that fine-tuning the vulnerability injection agent using Supervised Fine-Tuning (SFT) led to consistent and significant performance gains, particularly on complex datasets. The study also highlighted the critical role of the agentic workflow design; performance consistently improved as more specialized agents were added to the pipeline. Moreover, domain-specialized AI models (like Qwen2.5-Coder) significantly outperformed general-purpose large language models (like Llama-4-Maverick), underscoring the value of specialized AI for structured tasks like vulnerability injection.
The success of AVIATOR stems from several key design choices: its agentic decomposition of complex tasks, targeted and efficient model-level adaptation, iterative refinement capabilities through feedback loops, and the integration of traditional static analysis tools for robust validation. These elements combine to create a system that can generate high-quality, accurately labeled, and diverse datasets, which are crucial for advancing AI-driven software security.
Also Read:
- Enhancing Code Vulnerability Detection with Counterfactual Data Augmentation
- JailExpert: A New Framework for Automated LLM Jailbreaking Through Experience
In conclusion, AVIATOR represents a significant advancement in automated vulnerability injection, offering a scalable solution to the persistent challenge of data scarcity in software security research. By mimicking human expert reasoning and leveraging advanced AI techniques, it promises to enhance the training of robust and effective vulnerability detection models. For more in-depth information, you can refer to the original research paper: AI Agentic Vulnerability Injection And Transformation with Optimized Reasoning.
