TLDR: A new research paper from SAP introduces a scalable and cost-efficient framework for Graph-based Retrieval-Augmented Generation (GraphRAG) in enterprise settings. The framework features two core innovations: a dependency-based knowledge graph construction pipeline that eliminates reliance on LLMs, and a lightweight graph retrieval strategy using hybrid query node identification and one-hop traversal. Evaluated on SAP datasets, the system shows significant performance improvements over traditional RAG baselines and demonstrates that dependency-based graph construction achieves comparable results to LLM-generated graphs while drastically reducing costs and improving scalability, making GraphRAG practical for large-scale enterprise applications.
In the rapidly evolving landscape of artificial intelligence, Retrieval-Augmented Generation (RAG) has emerged as a powerful technique to enhance large language models (LLMs) by grounding their responses in external knowledge. This approach helps reduce inaccuracies and keeps models updated without costly retraining. However, traditional RAG systems often struggle with complex queries that require reasoning across multiple documents, a common challenge in large enterprise environments.
Addressing these limitations, a new research paper titled “Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems” introduces a groundbreaking framework for Graph-based RAG (GraphRAG). Authored by Congmin Min, Rhea Mathew, Joyce Pan, Sahil Bansal, Abbas Keshavarzi, and Amar Viswanathan Kannan from SAP, this work tackles the significant computational costs and latency issues typically associated with deploying GraphRAG at an enterprise scale.
The Core Problem with Traditional RAG and GraphRAG
While standard RAG excels at straightforward questions, it falters when faced with queries demanding multi-hop reasoning – questions that require connecting information from several different sources. Imagine trying to understand complex policy dependencies or legacy code migration impacts; these often span numerous documents and involve intricate relationships. Traditional RAG tends to return isolated snippets, making it difficult for LLMs to synthesize a coherent, logical answer.
GraphRAG offers a solution by building a structured knowledge graph from source documents. This graph represents entities (like people, places, or concepts) as nodes and their relationships as edges. At query time, the system can retrieve not just passages but also relevant subgraphs, allowing it to trace logical connections and build chains of evidence for more comprehensive answers.
However, deploying GraphRAG in large organizations comes with its own set of hurdles: the high computational cost of building these knowledge graphs, scalability issues as document collections grow, and the latency involved in querying large graphs in real-time.
Innovations for Scalable GraphRAG
The SAP researchers propose two key innovations to overcome these challenges:
-
Dependency-Based Knowledge Graph Construction: Traditionally, building knowledge graphs often relies on LLMs, which can be expensive and slow. This new framework completely eliminates LLM reliance for graph construction. Instead, it uses industrial-grade Natural Language Processing (NLP) libraries and a technique called dependency parsing. Dependency parsing analyzes the grammatical structure of sentences to identify how words relate to each other (e.g., subject-verb-object relationships), allowing for efficient extraction of entities and their connections. This significantly reduces cost and improves scalability.
-
Lightweight Graph Retrieval Strategy: To ensure quick responses, the system employs an efficient retrieval method. It combines hybrid query node identification (finding relevant starting points in the graph) with a fast “one-hop traversal.” This means it primarily looks at direct connections from the identified starting points, keeping the search space manageable while still retrieving highly relevant information.
How It Works: A Simplified Pipeline
The system operates in two main phases: knowledge graph construction and efficient graph retrieval.
Knowledge Graph Construction: Documents are first parsed and chunked into manageable sections. Sentences are then segmented, and a crucial step involves filtering out sentences without verbs, as these are less likely to contain meaningful relationships. The core innovation lies in the “TripleExtractor,” which, instead of an LLM, uses a dependency parser to identify subject-relation-object triples (e.g., {SAP, launch, Joule_for_Consultants}). These triples are then normalized and loaded into a knowledge graph database.
Efficient Graph Retrieval: When a user asks a question, the system first identifies key entities in the query. It then performs a one-hop traversal from these entities in the knowledge graph to find related nodes and relations. These candidate relations and relevant text chunks are then re-ranked using vector similarity to select the most relevant context. Finally, this rich context, including chunks, relations, and query entities, is passed to an LLM to generate a focused and accurate answer.
Real-World Validation and Performance
The framework was tested on two SAP datasets related to legacy code migration: CCM Chat (question-answer pairs) and CCM Code Proposal (legacy code examples). The results were compelling. The GraphRAG system showed significant improvements over traditional RAG baselines, with up to 15% improvement based on LLM-as-Judge metrics and 4.35% improvement using RAGAS metrics.
Crucially, the dependency-based construction approach achieved 94% of the performance of LLM-generated knowledge graphs (61.87% vs. 65.83% in weighted average score) while drastically cutting down on cost and improving scalability. For instance, LLM-based graph construction for a large workload could take over 60 days, whereas the dependency-based approach is far more efficient.
Qualitative analysis further highlighted the system’s strength. For example, when asked about handling custom code referencing a specific table (VBBS) after an S/4HANA conversion, the GraphRAG system successfully extracted content explicitly mentioning “VBBS” and relevant solutions, unlike the dense retriever. In code migration tasks, the GraphRAG-generated code exhibited fewer hallucinations and faulty definitions.
Also Read:
- CyberRAG: Enhancing Cyber Attack Detection with Agent-Powered AI
- Evaluating RAG Pipelines for Open Radio Access Networks: A Comparative Study
Looking Ahead
This research demonstrates the practical feasibility of deploying GraphRAG systems in real-world, large-scale enterprise applications without incurring prohibitive resource requirements. It paves the way for more practical, explainable, and domain-adaptable retrieval-augmented reasoning. While the dependency parsing approach is highly efficient, future work will explore its ability to capture more complex, context-dependent relations and its generalizability beyond SAP-specific domains.
For more in-depth information, you can read the full research paper here.
