TLDR: Foundation-Sec-8B-Instruct is a new large language model (LLM) specifically trained for general-purpose cybersecurity dialogue. Built on Foundation-Sec-8B, it combines deep domain knowledge with instruction-following and conversational capabilities. Evaluations show it outperforms Llama 3.1-8B-Instruct on cybersecurity tasks and is competitive with GPT-4o-mini, aiming to be a key assistant for cybersecurity professionals.
Large language models (LLMs) have made significant strides across various fields, but their full integration into cybersecurity applications has faced challenges. These hurdles include a scarcity of general-purpose cybersecurity data, the complexity of representing security concepts, and crucial safety and regulatory concerns. To address these issues, Foundation AI–Cisco Systems Inc. previously introduced Foundation-Sec-8B, an LLM specifically designed for cybersecurity, suitable for fine-tuning on specialized tasks.
However, the initial Foundation-Sec-8B model was not built for interactive chat or instruction-following. This new report introduces Foundation-Sec-8B-Instruct, a model specifically trained for general-purpose cybersecurity dialogue. This advanced model builds upon the domain-specific knowledge of Foundation-Sec-8B, enhancing it with instruction-following capabilities, conversational fluency, and alignment with human preferences to deliver high-quality and relevant responses.
Comprehensive evaluations demonstrate that Foundation-Sec-8B-Instruct surpasses Llama 3.1-8B-Instruct across a range of cybersecurity tasks, while also matching its performance in general instruction-following. Furthermore, it proves competitive with GPT-4o-mini in cyber threat intelligence and instruction-following scenarios. The developers envision Foundation-Sec-8B-Instruct becoming an essential tool for cybersecurity professionals in their daily work.
The model’s development involved a detailed analysis of cybersecurity data within existing open-source post-training datasets, identifying both the presence and distribution of relevant content. This analysis revealed that while some datasets contain minimal security content, others, like WizardLM Evol Instruct and Nemotron SFT Chat, have non-trivial proportions. The team also conducted a thorough contamination analysis to ensure the integrity of their evaluations, identifying and mitigating overlaps with benchmark data in training sets.
The training process for Foundation-Sec-8B-Instruct involved a combination of supervised fine-tuning (SFT) and reinforcement learning (RL) techniques, specifically Direct Preference Optimization (DPO). The focus was not on imparting new cybersecurity knowledge during post-training, but rather on refining instruction-following and aligning with human preferences, relying on the base model’s pre-trained knowledge. High-quality synthetic data, generated through a pipeline focusing on rejection sampling, difficulty grading, and automated verification, played a crucial role. Human preference testing further bolstered the preference tuning data, ensuring diverse and balanced datasets for robust generalization and knowledge retention.
In terms of performance, Foundation-Sec-8B-Instruct achieved state-of-the-art results on CTIBench-RCM, outperforming larger models like GPT-4o-mini and Llama 3.1-70B-Instruct. It also showed strong performance on other cybersecurity benchmarks such as CTIBench-MCQA, CyberMetric-500, SecBench, and SecEval. Beyond cybersecurity, the model excels in instruction-following and human-preferred output among cybersecurity LLMs, demonstrating a significantly higher win-rate on AlpacaEval 2 and strong performance on IFEval. It also maintains comparable performance to peer models in grade school math, mathematical reasoning, and coding tasks.
While Foundation-Sec-8B-Instruct has not undergone dedicated safety alignment beyond basic instruction-tuning, it demonstrated satisfactory performance on HarmBench, rejecting or safely responding to 92% of malicious examples. The researchers recommend pairing the model with additional safety layers, such as LlamaGuard, for production use cases, which increased the refusal rate on malicious requests to nearly 100% in evaluations. A detailed system prompt is also provided to enhance user experience and safety.
Also Read:
- Securing LLMs: A Dual Approach to Combat Prompt Injection and Data Leaks
- Safeguarding Large Language Models: A Deep Dive into Data Security Risks and Defenses
The release of Foundation-Sec-8B-Instruct marks a significant step towards meeting the growing demand for specialized cybersecurity LLMs with robust zero-shot performance across diverse contexts. This model aims to advance LLM integration in cybersecurity and become a daily assistant for many professionals. For more technical details, you can refer to the full report: Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report.
