TLDR: ALIGNed-LLM is a new framework that improves the factual accuracy of large language models (LLMs) by integrating Knowledge Graph Embeddings (KGEs). It uses a trainable projection layer to align structured knowledge from KGEs with LLM text embeddings, helping models reduce hallucinations and distinguish similar entities. Tested on various datasets, including a real-world financial KG, ALIGNed-LLM consistently showed significant improvements in question-answering performance compared to fine-tuned baselines.
Large language models (LLMs) such as GPT-4, Gemini, and Claude have revolutionized various natural language processing tasks, from answering questions to generating dialogue and summarizing text. However, a significant challenge remains: their tendency to ‘hallucinate,’ or generate factually incorrect information. This issue is particularly problematic in fields demanding high accuracy, like finance.
A promising solution to this problem involves integrating Knowledge Graphs (KGs) into language models. KGs are structured repositories of real-world knowledge, representing entities and their relationships in an organized, reliable, and often domain-specific manner. By providing LLMs with access to this structured information, their reasoning processes can be grounded in factual contexts, significantly enhancing accuracy and relevance.
Researchers have introduced a novel approach called ALIGNed-LLM, a simple yet effective method designed to improve the factual accuracy of language models. This approach draws inspiration from models like LLaVA, which combine visual and textual information. ALIGNed-LLM focuses on infusing knowledge from KGs directly into the ‘latent space’ of language models, which is essentially the internal representation where the model processes information.
How ALIGNed-LLM Works
The core of ALIGNed-LLM involves using pre-trained Knowledge Graph Embedding (KGE) models, such as TransE. These KGE models learn vector representations (embeddings) of entities and relationships within a KG, capturing their semantic and structural properties. In the ALIGNed-LLM framework, these KGE models are ‘frozen,’ meaning their learned embeddings are used as-is without further modification during the main training process.
When a query is given to ALIGNed-LLM, it identifies a ‘reference entity’ within that query. The corresponding embedding for this entity is retrieved from the pre-trained KGE model. This entity embedding is then passed through a special ‘trainable projection layer.’ This layer’s job is to transform the entity embedding so that it aligns with the language model’s own text embedding space. This alignment is crucial because it allows the structured KG knowledge to be seamlessly integrated with the textual understanding of the LLM.
Once aligned, the transformed entity embedding is combined with the language model’s textual embedding of the query. This combined input, now rich with both textual context and specific factual information from the KG, is then fed into the language model to generate the final answer. This process enables the LLM to better distinguish between similar entities, reducing ambiguity and mitigating hallucinations.
Training the Model
The training of ALIGNed-LLM occurs in two stages. First, a ‘feature alignment’ stage focuses solely on training the projection layer, ensuring that the KG entity embeddings are correctly mapped into the LLM’s internal representation space, while the LLM’s core weights remain unchanged. The second stage involves ‘end-to-end fine-tuning,’ where both the projection layer and the LLM’s output layer are jointly trained. This combined training helps the model generate accurate responses that are grounded in both the input text and the factual knowledge from the KG.
Also Read:
- Unpacking LLM Performance in SPARQL Query Generation: The Role of Knowledge and Memorization
- KROMA: Enhancing Ontology Matching with Context-Aware Language Models
Real-World Impact and Results
The effectiveness of ALIGNed-LLM was tested across various question-answering datasets, including general knowledge datasets like Wikidata and YAGO3-10, and a complex, real-world financial dataset from a large central bank in Europe, focusing on company ownership. The results were compared against standard fine-tuned versions of popular open-source LLMs like TinyLlama-1.1B, Vicuna-13B, and Mistral-7B.
The experiments showed consistent and significant improvements in factual accuracy across all datasets when using the ALIGNed approach. For instance, the ALIGNed Mistral-7B model demonstrated a substantial gain in exact match accuracy on the Wikidata dataset. Similarly, the ALIGNed TinyLlama-1.1B showed significant improvements on the Company Ownership dataset, highlighting its ability to enhance precision in identifying exact answers in a demanding domain. The choice of projection layer (simple linear or more complex multi-layered perceptron) was found to depend on the specific structure of the knowledge graph, with linear projections sometimes performing better on sparser graphs.
In conclusion, ALIGNed-LLM offers a robust framework for integrating knowledge graph embeddings into language models. By aligning entity and text embeddings through a trainable projection layer, it effectively enhances the factual accuracy and reasoning capabilities of LLMs, significantly reducing hallucinations and knowledge gaps. This approach not only bridges probabilistic inference with grounded knowledge but also offers practical advantages such as adaptability to domain-specific knowledge and ease of knowledge updates, requiring only embedding updates. For more technical details, you can refer to the full research paper: Aligning Knowledge Graphs and Language Models for Factual Accuracy.
