TLDR: A new research paper introduces a novel method for improving cyber incident response using a lightweight large language model (LLM). The method employs a three-step process: fine-tuning the LLM on a large dataset of incident logs, augmenting its knowledge with real-time threat intelligence, and using a decision-theoretic planning algorithm to select the most effective response actions with reduced hallucination. This approach achieves significantly shorter recovery times (up to 22% faster) compared to frontier LLMs, is lightweight enough to run on commodity hardware, and generalizes well across various incident types, offering a practical and efficient AI-driven solution for cybersecurity.
Cyberattacks are a constant threat, and responding to them quickly and effectively is crucial for organizations. Traditionally, incident response has been a largely manual and time-consuming process, often relying on security operators and predefined playbooks. While these methods can be effective, they are slow, labor-intensive, and struggle to keep up with the rapidly evolving landscape of cyber threats.
Recent advancements in large language models (LLMs) have opened new avenues for assisting security operators. LLMs possess vast amounts of security knowledge, making them potential tools for generating effective response actions. However, current approaches often depend on expensive, powerful LLMs and are prone to ‘hallucinations’ – generating plausible but incorrect or irrelevant information.
A new research paper, titled “Incident Response Planning Using a Lightweight Large Language Model with Reduced Hallucination,” introduces a novel method that addresses these limitations. This approach aims to provide decision support for incident response using a lightweight LLM, making it more accessible and reliable. You can find the full paper here.
A Three-Step Approach to Smarter Incident Response
The method proposed by Kim Hammar, Tansu Alpcan, and Emil C. Lupu involves three key steps:
1. Fine-tuning a Lightweight LLM: Unlike relying on massive, general-purpose LLMs, this method starts by fine-tuning a smaller, more specialized LLM. This offline process involves training the model on a large dataset of 68,000 incident logs, paired with correct response plans and detailed reasoning steps. This training helps the LLM learn the typical patterns and logical dependencies of incident handling, significantly reducing its tendency to hallucinate.
2. Information Retrieval (Retrieval-Augmented Generation – RAG): Even a fine-tuned LLM might lack information on the very latest threats or vulnerabilities. To overcome this, the method incorporates an online information retrieval step. When an incident occurs, the system automatically extracts indicators of compromise (like hostnames or vulnerability IDs) from system logs. It then uses this information to pull relevant, up-to-date threat intelligence from external sources, such as threat intelligence APIs and vulnerability databases. This fresh information is then fed to the LLM, allowing it to generate responses that are grounded in current threat data.
3. Decision-Theoretic Planning: Instead of simply accepting the LLM’s first suggested action, the method employs a sophisticated planning procedure. The LLM generates multiple potential response actions. For each candidate action, the system uses the LLM itself to simulate possible outcomes and estimate the expected time to recover from the incident. The action that is predicted to lead to the shortest recovery time is then selected. This ‘lookahead planning’ acts as a self-verification mechanism, further reducing the likelihood of hallucinated or ineffective actions.
Also Read:
- Automating Cybersecurity Playbook Conversion to CACAO Format with AI
- Training AI to Challenge AI: A Multi-Turn Red Teaming Strategy for LLMs
Performance and Advantages
The researchers evaluated their method using real-world log data from various cyber incidents. The results are compelling:
- Faster Recovery: The method achieved up to 22% shorter recovery times compared to leading frontier LLMs like Google’s Gemini 2.5 and OpenAI’s O3.
- Lightweight and Accessible: Unlike large, resource-intensive LLMs, this lightweight model can run efficiently on standard hardware, making it more practical for deployment.
- Generalizability: The approach demonstrated its ability to generalize across a wide range of incident types and necessary response actions.
- Open Source: The researchers have made their fine-tuned LLM, dataset, and source code publicly available, fostering further research and development in the field.
When compared to other established methods, this new approach stands out. While reinforcement learning methods like Proximal Policy Optimization (PPO) can achieve similar performance, they require extensive, incident-specific training, which is often impractical in real-time scenarios. This LLM-based method, however, does not require such incident-specific training. Furthermore, it offers significant advantages over traditional response playbooks by generating more precise, context-specific, and actionable response plans without relying on constant manual configuration by security experts.
In conclusion, this research presents a significant step forward in leveraging artificial intelligence for cybersecurity. By combining fine-tuning, real-time information retrieval, and intelligent planning, it offers a more efficient, reliable, and practical solution for incident response, ultimately helping organizations recover faster from cyberattacks.
