TLDR: LATTE is a new AI framework that learns client behaviors from financial transactions by aligning efficient sequence embeddings with semantic descriptions generated by large language models (LLMs). It significantly reduces computational costs compared to direct LLM use, achieving state-of-the-art performance on banking tasks like churn and gender prediction, making it practical for real-world financial applications.
Understanding client behavior from their financial transaction history is a crucial task for banks. This involves creating “embeddings” – numerical representations – of clients based on their past activities. These embeddings are vital for various applications like predicting if a client might leave (churn prediction), assessing risk, or offering personalized services.
Traditionally, large language models (LLMs) have shown great promise in understanding complex data. However, directly applying LLMs to long sequences of financial events, such as hundreds of transactions per client, becomes computationally very expensive and slow. This is because LLMs require a lot of processing power and can struggle with very long inputs, making them impractical for real-world banking systems that need to process millions of records quickly.
Introducing LATTE: A Smarter Approach to Client Embeddings
To overcome these challenges, researchers have proposed a new framework called LATTE (Learning Aligned Transactions and Textual Embeddings for Bank Clients). This innovative approach uses a technique called contrastive learning to align raw financial event embeddings with semantic embeddings generated by powerful, pre-trained LLMs. Instead of feeding entire transaction histories to an LLM, LATTE summarizes key behavioral features into short, descriptive prompts. These prompts are then fed to a “frozen” LLM (meaning it’s used as-is, without further training) which generates natural language summaries of the client’s behavior.
In parallel, a lightweight encoder processes the original transaction sequences to create sequence embeddings. The magic happens next: LATTE uses a contrastive loss function to train the lightweight encoder to align its sequence embeddings with the semantic embeddings from the LLM-generated text. This alignment process allows the system to capture the rich semantic understanding of the LLM without incurring the high computational cost of processing full sequences with the LLM during inference.
How LATTE Works in Practice
The LATTE pipeline involves three main stages. First, raw transaction sequences are analyzed to compute summary features, such as transaction frequency, merchant diversity, and income-expense patterns. These statistics are then transformed into a structured textual prompt. Second, this prompt is given to an instruction-tuned LLM, which generates a natural language description of the client’s financial behavior. Third, this textual description is converted into a text embedding by a separate text encoder. Simultaneously, the original transaction sequence is processed by a GRU-based sequence encoder. Finally, these two types of embeddings – the sequence embedding and the text embedding – are aligned using one of three contrastive loss functions, effectively teaching the sequence encoder to understand the “meaning” of the transactions as described by the LLM.
LATTE offers two main modes for deployment: LATTE-S (standalone encoder) and LATTE (combined encoder). LATTE-S is highly efficient, using only the lightweight sequence encoder at inference time, making it suitable for latency-sensitive environments. The full LATTE combines both textual and structural representations for even higher accuracy.
Impressive Results on Real-World Data
The researchers evaluated LATTE on three real-world banking datasets for tasks like churn prediction, age group classification, and gender prediction. The results showed that LATTE consistently outperformed existing state-of-the-art methods. For instance, LATTE achieved significant improvements in gender prediction (6.1%), age group classification (3.0%), and churn prediction (2.0%) compared to baselines.
Crucially, LATTE-S demonstrated remarkable efficiency, processing over 160 samples per second per GPU with a compact model size of just 4.4 million parameters. This makes it significantly faster than other LLM-based models, which often process only a few users per second, while still maintaining high accuracy.
Ablation studies further highlighted the importance of LLM-generated descriptions for performance gains. The choice of the LLM for description generation and the text embedding model also played a significant role, with larger, more capable models generally yielding better results. Using structured statistical summaries for prompting the LLM proved more effective than raw transaction serialization.
Also Read:
- TokenCast: Bridging Numerical and Textual Data for Advanced Time Series Forecasting
- Unlocking Deeper Insights in Time Series Data with TimeMKG’s Knowledge-Infused Approach
Looking Ahead
LATTE represents a significant step forward in learning robust and semantically rich representations from financial event sequences. By intelligently combining the power of LLMs with efficient sequence encoders, it provides a scalable and high-performing solution for critical banking applications. The framework’s ability to derive insights without relying on extensive manual supervision is particularly valuable in industries where labeled data is scarce.
The full research paper can be found here: LATTE Research Paper.
