TLDR: RAGRank is a novel defense mechanism against poisoning attacks in Retrieval-Augmented Generation (RAG) systems used for Cyber Threat Intelligence (CTI). It leverages a PageRank-derived authority score, enhanced with time decay and author credibility, to filter out malicious or untrustworthy information. By focusing on source credibility rather than just content, RAGRank improves the accuracy of LLM responses in CTI contexts, as demonstrated in experiments with poisoned datasets and CTI misinformation scenarios.
Large Language Models (LLMs) have become indispensable tools across many industries, and their application in Cyber Threat Intelligence (CTI) is particularly transformative. By analyzing vast amounts of unstructured data, LLMs can help identify threats, extract crucial indicators of compromise (IoCs), and provide actionable insights to cybersecurity professionals. However, LLMs have a significant limitation: they lack real-time access to current, domain-specific knowledge and can sometimes ‘hallucinate’ or generate incorrect information when data is scarce.
To overcome this, Retrieval-Augmented Generation (RAG) has emerged as a leading architectural pattern. RAG systems dynamically pull relevant, up-to-date information from curated knowledge bases or external sources, effectively grounding the LLM’s responses in verifiable evidence. This allows LLMs to perform tasks like summarizing threat actor profiles or explaining complex attack techniques with greater accuracy and relevance.
The Challenge of Poisoning Attacks
While RAG enhances LLM capabilities, it also introduces a new vulnerability: poisoning attacks. Adversaries can inject malicious information—such as falsified threat reports, manipulated IoCs, or misleading vulnerability data—into the knowledge bases that RAG systems rely on. Since CTI is often a shared resource, such misinformation can corrupt entire security pipelines, leading to wasted resources, ineffective mitigations, or even obscuring genuine attacks. Traditional defenses against RAG poisoning often focus on analyzing the content itself, but these can fail in CTI contexts because new cyber threats emerge constantly, and sophisticated attackers can mimic legitimate formats and terminology.
Introducing RAGRank: A Credibility-Based Defense
To address this critical issue, researchers have proposed a novel defense mechanism called RAGRank. Instead of solely scrutinizing what the information says, RAGRank focuses on where it originates and how it propagates. This approach draws inspiration from search engine ranking tactics, particularly Google’s PageRank algorithm, which measures the prestige and contextual support for webpages based on inbound hyperlinks.
RAGRank applies source credibility algorithms to document corpora, using a PageRank-derived authority score. The system first builds a ‘citation network’ from a document corpus. This network can be constructed in a few ways: using explicit citations, inferring citations based on topical commonality between documents (often with the help of an LLM like Gemma 2-27B), or by extracting and linking individual claims within documents. The inferred citation method, for instance, determines if one document can be considered a continuation of another’s discussion, also factoring in temporal aspects (newer documents citing older ones).
How RAGRank Calculates Authority
Once the graph is built, RAGRank calculates an authority score for each document. This score is then enhanced with additional measures:
- Time-Decayed Rank: To ensure that newer, potentially critical threat information isn’t overlooked, RAGRank applies a time decay factor. Older documents’ scores are scaled down based on their age, preventing outdated information from retaining undue influence.
- Author Credibility: The system also considers the reputation of authors. A new document from a historically reputable author may receive an increased initial authority score, while content from less credible sources is restricted.
These components are combined and normalized to produce the final RAGRank score, ranging from 0 to 1, for each document.
Integrating RAGRank into Retrieval
In a RAG pipeline, RAGRank is used in a two-pass ranking strategy. First, the system retrieves a larger set of documents (e.g., top-2k) based on their semantic similarity to the user’s query. In the second pass, these candidates are re-ranked using their RAGRank score. Only the top-k documents from this re-ranked list are then selected as context for the LLM. This ensures that only semantically relevant documents are considered, while still prioritizing trustworthy and influential sources. This method prevents high-authority but less relevant documents from overshadowing more relevant but lower-authority ones.
Also Read:
- Dynamic Defense for LLM-Powered Multi-Agent Systems
- Structured Answers: Interpretable QA Using Knowledge Graphs
Experimental Validation
Experiments demonstrated RAGRank’s effectiveness. On poisoned versions of the standardized MS MARCO question answering dataset, RAGRank improved the LLM’s accuracy by about 10-15% compared to systems without poisoning protection. In proof-of-concept CTI scenarios, RAGRank successfully countered misinformation. For example, when queries about known Advanced Persistent Threats (APTs) were met with poisoned blog posts, RAGRank assigned low authority scores to the malicious content, ensuring the LLM relied on high-authority sources like Wikipedia. In another scenario, an attacker attempted to ‘front-run’ information about malicious infrastructure by posting a blog claiming a domain was benign. Without RAGRank, the LLM would have been poisoned. With RAGRank, the system correctly identified the lack of authoritative information, informing the user that the domain’s status was unknown due to unreliable sources.
This research highlights the importance of source credibility in securing AI-powered CTI systems. RAGRank offers a robust defense against poisoning attacks, ensuring that LLMs are grounded in trustworthy information, even in the face of sophisticated adversarial tactics. For more details, you can refer to the full research paper: RAGRank: Using PageRank to Counter Poisoning in CTI LLM Pipelines.
