TLDR: This research introduces a graph-based Retrieval-Augmented Generation (RAG) system that uses Large Language Models (LLMs) to answer questions about energy efficiency. It automatically extracts knowledge from documents to build a Knowledge Graph, which is then used to provide accurate, multilingual answers. Validated by domain experts, the system achieved an overall answer validity score of 75.2%, demonstrating its potential to offer tailored recommendations and overcome the limitations of traditional LLMs in domain-specific contexts.
In today’s world, the focus on energy efficiency is more critical than ever, driven by the need to reach net-zero emissions and promote sustainable practices. As individuals and organizations strive to adopt eco-friendly behaviors and optimize energy consumption, access to accurate and reliable information becomes paramount. This is where advanced Artificial Intelligence (AI) systems, particularly Large Language Models (LLMs), can play a significant role.
However, LLMs, while powerful, sometimes struggle to provide factual answers, especially when dealing with highly specific or domain-specific information, local regulations, or socioeconomic contexts. This limitation can lead to what is known as ‘hallucinations’ – made-up or nonsensical answers that erode user trust and spread misinformation. To address this challenge, researchers are turning to Retrieval-Augmented Generation (RAG) systems, which combine the generative power of LLMs with a robust knowledge base.
A New Approach to Energy Efficiency Question Answering
A recent research paper, titled “A Graph-based RAG for Energy Efficiency Question Answering” by Riccardo Campi, Nicolò Oreste Pinciroli Vago, Mathyas Giudici, Pablo Barrachina Rodriguez-Guisado, Marco Brambilla, and Piero Fraternali, introduces an innovative graph-based RAG architecture designed to provide accurate and multilingual answers to energy efficiency questions. This system aims to offer users tailored recommendations by integrating domain-specific knowledge from guidance and regulatory documents in the energy field. You can read the full paper here.
How the System Works: A Three-Part Architecture
The proposed system operates in three main stages:
1. Knowledge Extraction: The process begins by feeding domain-specific documents, such as PDFs or web pages containing information on energy consumption, regulations, and incentives, into the system. These documents are first cleaned of unnecessary elements and then divided into smaller, manageable chunks. An LLM-based algorithm then parses these chunks to automatically extract entities (objects) and relationships (actions), forming entity-relationship-entity triples. For example, it might extract “LED bulbs – offer – energy efficiency.” Domain experts can guide this extraction process to ensure adherence to specific terminologies or ontologies. Finally, the extracted entity and relationship names are standardized for consistency.
2. Knowledge Base: The extracted triples are used to build a Knowledge Base, which primarily consists of a Knowledge Graph (KG) and auxiliary tables. The KG is a network of nodes and edges, where nodes represent entities (like “LED bulbs” or “Italian regulations”) and edges represent relationships between them. This graph is populated only once or when updates to the information are needed. Auxiliary tables store user-specific metadata, such as location and preferences, allowing for personalized answers. To enable efficient searching, embedding vectors (numerical representations) are computed for entity and relationship names, as well as for the chunked text content.
3. Retrieval & Generation: When a user asks a question, the system first extracts relevant entities from the question using the same LLM-based algorithm. It then compares these entities with those in the Knowledge Graph using a similarity measure to identify the most relevant ones. A local reasoning process then starts from these identified entities, navigating the graph to gather all pertinent information. This retrieved information, along with the original question and user metadata, is then fed to an LLM, which generates a comprehensive answer, augmented with citations to the original source documents. If no relevant entities are found, the system indicates that no results exist for the question.
Validation and Results
To assess the system’s effectiveness, a human-based validation experiment was conducted using a dataset of 101 energy efficiency question-answer pairs in both Italian and English. Domain experts evaluated the answers based on the RAGAs framework, which assesses faithfulness (grounded in context), answer relevance, and context relevance. The system was also integrated into ENERGENIUS Guru, a Decision Support System for Energy Efficiency.
The results were promising: the system achieved an overall answer validity score of 75.2%. It performed well in both Italian (77.4%) and English (73.0%), demonstrating strong multilingual capabilities with only a 4.4% accuracy loss due to translation. The system performed slightly better on country-agnostic questions (78.1%) compared to country-specific ones (73.4% for Italy, 71.2% for Switzerland), likely because country-specific laws and articles often contain more complex, temporally or spatially constrained information. An ablation experiment, where the RAG component was removed, highlighted the critical need for retrieval technology, as the LLM-only architecture frequently provided inaccurate or irrelevant answers to domain-specific questions.
Also Read:
- ET2RAG: Boosting Language Model Accuracy with Smart Retrieval and Consensus
- GraphChain: A New Approach for Large Language Models to Analyze Complex Graph Data
Conclusion
This research underscores the significant potential of combining LLMs with a Knowledge Graph-enhanced RAG architecture for energy efficiency. The system’s ability to automatically extract knowledge, perform local reasoning, and provide accurate, contextually rich, and multilingual answers represents a substantial step forward in making complex energy information accessible and actionable for users. While this is a preliminary study, it lays a strong foundation for future enhancements and broader language testing within the ongoing ENERGENIUS project.
