TLDR: SLiNT is a novel framework that improves knowledge graph completion by integrating structural information into large language models. It addresses challenges like missing data and confusing meanings through three modules: Structure-Guided Neighborhood Enhancement (SGNE) for enriching sparse entities, Dynamic Hard Contrastive Learning (DHCL) for resolving semantic ambiguity, and Gradient-Decoupled Dual Injection (GDDI) for efficient, token-level structural context injection. Experiments show SLiNT achieves superior performance and robustness on standard benchmarks, making it a powerful tool for accurate link prediction.
Knowledge graphs (KGs) are fundamental for representing real-world facts as structured triples, like (head entity, relation, tail entity). They power various applications, from question answering to recommendation systems. However, real-world KGs are often incomplete, leading to the crucial task of knowledge graph completion (KGC) – predicting these missing entities or relations.
Traditional KGC methods, which learn low-dimensional embeddings for entities and relations, often struggle with entities that have sparse local connections. While some extensions incorporate textual features or graph-aware context, they still face challenges with generalization and semantic discrimination. More recently, large language models (LLMs) have emerged as a new paradigm for KGC, leveraging their semantic understanding to generate missing entities from textual queries. However, LLMs often have limited exploitation of structural signals, leading to issues like structural sparsity (where insufficient structural cues cause the model to hallucinate answers) and semantic ambiguity (where models might choose semantically similar but structurally incorrect alternatives).
Introducing SLiNT: A Structure-Aware Approach
To address these persistent challenges, researchers Mengxue Yang, Chun Yang, Jiaqi Zhu, Jiafan Li, Jingqi Zhang, Yuyang Li, and Ying Li propose SLiNT (Structure-aware Language model with Injection and coNtrastive Training). SLiNT is a modular framework designed to inject KG-derived structural context into a frozen LLM backbone, using lightweight LoRA-based adaptation for robust link prediction. You can read the full paper here: SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion.
SLiNT integrates three key components:
1. Structure-Guided Neighborhood Enhancement (SGNE): This module tackles structural sparsity. It retrieves “pseudo-neighbors” – entities that are structurally similar in the KG embedding space – to enrich the contextual representations of sparsely connected entities. By fusing these pseudo-neighbors with attention mechanisms, SGNE helps build richer, more informative representations.
2. Dynamic Hard Contrastive Learning (DHCL): To combat semantic ambiguity, DHCL introduces fine-grained supervision. Instead of simply comparing entity pairs, it interpolates between the query and structure-derived prototypes to generate “hard positives” and “hard negatives.” This process simulates ambiguous decisions near structural margins, encouraging the model to distinguish structurally coherent answers from misleading, semantically similar distractors.
3. Gradient-Decoupled Dual Injection (GDDI): This mechanism bridges the gap between structural representations and language generation. GDDI injects the enhanced structural representations into the frozen LLM backbone at the token level through prompt-based augmentation and substitution. Crucially, it leverages lightweight LoRA adaptation, meaning only a small fraction of the LLM’s parameters are fine-tuned, maintaining efficiency while preserving the core LLM capabilities.
Performance and Robustness
Experiments conducted on two widely used KGC benchmarks, WN18RR and FB15k-237, demonstrate SLiNT’s effectiveness. The framework achieves superior or competitive performance compared to both embedding-based and generation-based baselines. For instance, SLiNT significantly improves MRR (Mean Reciprocal Rank) scores compared to vanilla LLaMA variants, highlighting the benefits of its structure-aware injection and contrastive learning.
An ablation study confirmed that each of SLiNT’s modules – SGNE, DHCL, and GDDI – contributes uniquely to the overall performance. Removing any one component led to a noticeable drop in accuracy, underscoring their integrated necessity.
Furthermore, SLiNT exhibits strong robustness under challenging conditions, such as limited training data or incomplete structural connectivity. It maintains high performance even when dealing with low-degree entities (those with very few connections), which are typically difficult for other models. This adaptability makes SLiNT a plug-and-play solution compatible with various structural encoders, allowing it to leverage different structural priors effectively.
Also Read:
- DEEPGRAPHLOG: Integrating Neural Networks and Symbolic Logic for Advanced AI Reasoning
- Bridging Language Models and Knowledge Graphs for Biomedical Understanding
Conclusion
SLiNT represents a significant advancement in knowledge graph completion. By explicitly integrating structural context and fine-grained supervision into large language models, it offers a robust and efficient framework for predicting missing links. Its ability to handle structural sparsity and semantic ambiguity, combined with its parameter-efficient adaptation, makes it a promising solution for scalable KGC in diverse real-world applications.
