TLDR: ReaL-TG is a new reinforcement learning framework that fine-tunes large language models (LLMs) to predict future connections (link forecasting) in dynamic, real-world networks (temporal graphs). It enables LLMs to generate human-readable explanations for their predictions and perform well on new, unseen graphs without retraining. The framework also introduces a novel evaluation system that assesses both prediction accuracy and the quality of the LLM’s reasoning, showing that a fine-tuned 4-billion parameter model can outperform much larger LLMs like GPT-5 mini.
Temporal graphs, which model interactions between entities over time, are crucial for understanding dynamic real-world scenarios like social networks and financial transactions. A key challenge in this field is link forecasting – predicting future connections based on past interactions. While traditional neural network approaches have shown strong performance, they often fall short in two critical areas: providing human-readable explanations for their predictions and adapting to new, unseen graphs without extensive retraining.
Recently, large language models (LLMs) have emerged as potential tools for graph reasoning, offering the inherent ability to generate explanations and generalize to new data. However, most existing LLM applications in this domain have been limited to static graphs or small, synthetic temporal graphs. Crucially, there has been a lack of robust methods to evaluate the quality of the reasoning processes generated by these LLMs.
Introducing ReaL-TG: Explainable Link Forecasting with LLMs
To address these limitations, researchers have developed Reasoning-Enhanced Learning for Temporal Graphs (ReaL-TG), a novel reinforcement learning framework designed to fine-tune LLMs for explainable link forecasting on real-world temporal graphs. Unlike previous efforts that might rely on textual attributes or small datasets, ReaL-TG is built and tested on anonymized real-world temporal graphs from the popular Temporal Graph Benchmark (TGB). This approach ensures that the model learns to reason purely from the graph’s structural dynamics, preventing potential data leakage from pre-training on semantic information.
The core of ReaL-TG involves fine-tuning an LLM using an outcome-based reward system. This system encourages the model to explore and discover effective reasoning strategies directly from the graph structure. It also pushes the LLM to produce clear, human-readable explanations that directly justify its predictions, aiming for both high predictive accuracy and logically sound reasoning.
How ReaL-TG Works
The framework operates by first selecting a relevant subgraph, called a Temporal Context Graph (Gc), for each prediction query. This selection is done using a specialized algorithm that focuses on historical interactions most pertinent to the query. The links within this Gc are then converted into text and combined with a natural language question to form a prompt for the LLM. The LLM processes this prompt, generates a prediction, and provides an explanatory reasoning trace. A customized reward function, based on the F1 score, guides the reinforcement learning process, rewarding accurate predictions while balancing precision and recall.
A New Standard for Evaluation
To comprehensively assess LLMs in temporal graph link forecasting, ReaL-TG introduces an innovative evaluation protocol. This protocol extends the traditional Mean Reciprocal Rank (MRR) with a ‘penalized MRR’ (pMRR), which specifically penalizes the LLM for over-generating predictions (i.e., predicting nodes that are not part of the ground truth). This helps to capture the risk of generating too many answers in a QA-style task.
Beyond prediction accuracy, the protocol also features an LLM-as-a-Judge system to evaluate the quality of the generated reasoning traces. This judge system assesses three critical criteria:
- Faithfulness: Whether the reasoning is supported by the input graph context, identifying factual hallucinations.
- Logical Consistency: Whether the reasoning follows a coherent and valid chain, detecting logical inconsistencies.
- Answer–Explanation Alignment: Whether the predicted answers are justified by the model’s own reasoning, revealing justification hallucinations.
Impressive Performance and Explainability
Experiments with ReaL-TG-4B, a model obtained by fine-tuning Qwen3-4B within this framework, demonstrate remarkable results. ReaL-TG-4B not only outperforms much larger and more advanced LLMs, including GPT-5 mini and Llama 3.3 70B, in prediction accuracy on both familiar and entirely new graphs, but it also generates high-quality explanations. Both the LLM-as-a-Judge system and human evaluators confirmed the quality of these explanations. The research paper detailing this work can be found here.
The study also highlights that while larger base models generally lead to better reasoning quality, the ReaL-TG framework significantly boosts the performance of even smaller models, enabling them to discover meaningful reasoning patterns. Human evaluations further validated the reliability of the LLM-as-a-Judge system, suggesting its potential for scalable assessment of LLM reasoning.
Also Read:
- Agentic Reinforcement Learning: Empowering LLMs as Autonomous Decision-Makers
- Smart Planning for LLM Agents: Balancing Speed and Expense
Future Directions and Impact
ReaL-TG represents a significant step forward in making LLMs effective and transparent for complex graph reasoning tasks. By enabling LLMs to provide both accurate predictions and meaningful explanations, this framework offers new insights into the dynamics of evolving networks. This has profound implications for high-stakes applications like fraud detection and network attack analysis, where interpretability is paramount. While LLMs still carry risks like hallucination, ReaL-TG demonstrates a promising approach to mitigate these issues and foster greater trust in AI-driven predictions.
