TLDR: DGP (Dual Granularity Prompting) is a novel framework designed to improve fraud detection using Graph-Enhanced Large Language Models (LLMs). It addresses the challenge of information overload in complex networks by providing LLMs with fine-grained details about the target entity while summarizing information from its neighbors into concise, coarse-grained prompts. This approach significantly enhances fraud detection accuracy and efficiency, demonstrating the potential of LLMs in analyzing intricate graph data.
Fraud detection is a critical challenge in today’s interconnected world, from identifying fake accounts on social media to spotting suspicious transactions in e-commerce. Traditional methods often struggle with the sheer volume and complexity of real-world data, which frequently involves intricate relationships between entities, best represented as graphs.
Recently, Large Language Models (LLMs) have shown immense promise in various AI tasks, and researchers are exploring ways to leverage their powerful reasoning capabilities for graph-based problems. The idea is to convert graph information into text-based prompts that LLMs can understand and process. However, a significant hurdle arises when dealing with heterogeneous graphs, where different types of nodes and relationships exist. In such scenarios, the ‘neighborhood’ of a target entity (i.e., its connected nodes) can expand exponentially, leading to prompts that are excessively long and filled with irrelevant information. This ‘information overload’ can dilute crucial signals from the target entity, making it harder for LLMs to accurately detect fraud.
Introducing Dual Granularity Prompting (DGP)
To address this challenge, researchers from the National University of Singapore and ByteDance Inc. have proposed a novel framework called Dual Granularity Prompting (DGP). This innovative approach tackles information overload by adopting a smart, two-tiered strategy: it preserves fine-grained, detailed textual information for the target node (the entity being evaluated for fraud) while summarizing the information from its neighbors into concise, coarse-grained text prompts.
DGP achieves this balance through tailored summarization techniques. For textual data, it employs a bi-level semantic abstraction, effectively condensing verbose neighbor content. For numerical features, it uses statistical aggregation to retain key insights. This dual-granularity design ensures that the LLM receives all necessary information without being overwhelmed by excessive detail.
How DGP Works
The DGP framework operates through three core modules. First, a node-level summarization process distills the essence of each node’s raw text into a brief, representative summary. This is done in a task-agnostic manner, meaning it doesn’t rely on specific fraud-related keywords, allowing for broader applicability.
Second, a diffusion-based metapath trimming method selects only the most structurally and semantically relevant neighbors along specific ‘metapaths’ (sequences of relationships in the graph). This step is crucial for filtering out noise and focusing on context that is truly indicative of fraud.
Finally, a metapath-level summarization module further aggregates the node-level summaries of these selected neighbors, creating a concise, informative representation for each type of relationship. Numerical features are also summarized through mean aggregation, providing complementary signals to the textual summaries.
These fine-grained target node details and coarse-grained neighbor summaries are then combined to construct structured prompts for the LLM. The LLM is then fine-tuned to predict whether a node is fraudulent or benign based on these carefully crafted prompts.
Also Read:
- Adaptive Prompt-tuning for Efficient Federated Learning
- Bridging Graph Data and Large Language Models with Quantized Tokens
Performance and Impact
Extensive experiments conducted on both public and industrial datasets demonstrate DGP’s superior performance. The framework operates within a manageable token budget, a critical factor for LLM efficiency, while significantly improving fraud detection accuracy. Notably, DGP boosted fraud detection performance by up to 6.8% (AUPRC) compared to state-of-the-art methods.
The research highlights that even highly coarse-grained summaries of neighbor information (as few as 10 tokens) are sufficient to enhance fraud detection, indicating that DGP is effective even in complex real-world graphs where detailed neighbor descriptions might be impractical. Interestingly, the study also found that task-agnostic summarization (generic summarization) performed better than task-aware summarization (summarization specifically focused on fraud signals), suggesting that a broader approach allows the LLM to discover more subtle fraud patterns.
DGP represents a significant step forward in leveraging Graph-Enhanced LLMs for fraud detection. By intelligently managing information flow and preventing attention dilution, it unlocks the full potential of LLMs to reason over both textual and structural information in complex, heterogeneous graphs. For more technical details, you can read the full research paper here.
