TLDR: Pref-LSTM is a new framework that combines a BERT-based classifier to detect user preferences with an LSTM-inspired memory module to store them. While the BERT classifier effectively identifies preferences, the LSTM memory module did not significantly improve LLM preference following in initial tests, pointing to areas for future research in scalable and lightweight LLM personalization.
Large Language Models (LLMs) are incredibly powerful for tasks like answering questions and writing, but they often struggle to remember user-specific preferences over long conversations. This limitation becomes more apparent as LLMs are used for personal assistants or educational tools, where remembering user likes and dislikes is crucial for personalized interactions.
A new research paper introduces Pref-LSTM, a novel framework designed to give LLMs a better memory for user preferences. The core idea is to efficiently identify and store user preferences without adding significant computational burden or requiring extensive fine-tuning of the LLM itself. Imagine telling an LLM you’re lactose intolerant, and then later asking for dessert recommendations – Pref-LSTM aims to ensure the LLM remembers your dietary restriction and avoids dairy suggestions.
How Pref-LSTM Works
Pref-LSTM operates in two main phases: memorization and inference. The memorization phase focuses on detecting user preferences and updating an internal memory. The inference phase uses this memory to guide the LLM’s responses.
The system first uses a BERT-based classifier to determine if a user’s statement contains a preference. This classifier was trained on a specially created dataset of preference and non-preference conversation turns. The researchers also explored a simpler, rule-based approach, but found the BERT-based classifier performed better, especially in identifying implicit preferences.
If a preference is detected, it’s encoded into a “memory embedding.” This embedding is then used to update the system’s internal memory, which is inspired by the gating principles of Long Short-Term Memory (LSTM) networks. LSTMs are known for their ability to selectively remember or forget information, which is key to managing dynamic user preferences.
For the LLM to use this memory, the memory embedding is transformed into a “soft prompt” and injected directly into the LLM’s input. This allows the LLM to generate personalized responses based on the stored preferences without needing to be retrained or extensively modified.
Training and Results
The training of Pref-LSTM involves two stages: training the preference classifier and training the LSTM memory controller. The BERT-based classifier showed strong performance in identifying preferences, particularly in formally structured language. However, its performance dropped with more casual language, suggesting a need for more diverse training data.
Interestingly, the LSTM-based memory controller, despite showing a decrease in training loss, did not lead to observable improvements in the LLM’s preference following during testing. The authors suggest several reasons for this, including computational limitations during training, the nature of the dataset used for the memory controller, and the method of injecting memory embeddings as soft prompts, which might sometimes act as noise rather than helpful context.
Also Read:
- A New Framework for Personalized AI: Integrating Cognitive Memory and Thought Processes
- Advancing LLM Personalization: A New Self-Supervised Approach to Reinforcement Learning from Human Feedback
Conclusion and Future Directions
While the LSTM memory module didn’t yield strong results in this initial study, the research highlights the potential of using preference filtering with LSTM-inspired gating for scalable user preference modeling. The BERT-based classifier proved reliable in identifying explicit and implicit user preferences, demonstrating a viable path for efficient preference detection.
Future work will focus on expanding the diversity of the preference classification dataset and exploring alternative ways for the LSTM memory embeddings to communicate with LLMs beyond simple soft-prompt injection. This research, detailed in the paper Dynamic LSTM-based Memory Encoder For Long-term LLM Interactions, paves the way for more personalized and context-aware LLM interactions.
