TLDR: A new dataset called ITDR has been created to help large language models (LLMs) better understand and perform recommendation tasks. It includes nearly 200,000 instances across various recommendation scenarios, significantly improving the performance of open-source LLMs and even outperforming larger closed-source models after fine-tuning. The dataset addresses the challenge of LLMs struggling with structured user behavior data by providing specific instructions for user-item interaction and understanding tasks.
Large Language Models (LLMs) have shown remarkable abilities in understanding and generating human language. However, when it comes to recommendation systems, these powerful models often face challenges. This is mainly because user behavior data, which is crucial for recommendations, is structured differently from natural language. While some methods use prompts to guide LLMs for recommendations, their understanding of these specific tasks has been limited, leading to less-than-ideal performance.
To bridge this gap, researchers have developed a new and comprehensive instruction tuning dataset called ITDR. This dataset is designed to significantly improve how LLMs handle recommendation tasks. ITDR is built from 13 different public recommendation datasets and includes approximately 200,000 instances. These instances are organized into 7 subtasks, which fall under two main categories: user-item interaction and user-item understanding.
What is ITDR?
ITDR stands for Instruction Tuning Dataset for Recommendation systems. It’s a large-scale dataset specifically created to train LLMs to better understand and perform tasks related to recommending items to users. The dataset covers a wide range of recommendation scenarios, ensuring that models trained on it can adapt to various real-world applications.
How ITDR is Structured
The dataset is divided into two core root tasks:
-
User-Item Interaction (UII): This focuses on how users interact with items. It includes subtasks like Rating Prediction (predicting a user’s rating for an item), Top-K Recommendation (suggesting the top K items a user might like), Cross-Domain Recommendation (using data from one area to recommend in another), and Next Item Recommendation (predicting what a user will interact with next in a sequence).
-
User-Item Understanding (UIU): This category delves into understanding both user attributes and item features. Subtasks here include User Attribute Prediction (inferring user characteristics like age or gender), Interest Recognition (identifying a user’s interests from their behavior), and Target User Identification (finding which users would be interested in a specific item).
Each subtask in ITDR comes with a carefully designed description to help the LLM understand the task’s goal, and the data is formatted using standardized templates. For some tasks where original datasets lacked specific annotations, advanced LLMs like DeepSeek-V3 were used to generate high-quality ground truth data.
Also Read:
- AI Models Master Community Resource Allocation Through Participatory Budgeting
- Streamlining Database Interaction: An End-to-End Text-to-SQL Framework with Automated Database Selection
Key Findings and Impact
Experiments conducted using ITDR showed impressive results. The dataset significantly boosted the performance of popular open-source LLMs such as GLM-4, Qwen2.5, Qwen2.5-Instruct, and LLaMA-3.2 across various recommendation tasks. This demonstrates that ITDR is effective and broadly applicable for enhancing LLM capabilities in this field.
The research also explored several important aspects:
-
Task Interactions: It was found that understanding user-item attributes (UIU tasks) can help models perform better on user-item interaction tasks (UII tasks), showing a positive transfer of knowledge. However, training only on UII data sometimes negatively impacted UIU performance, suggesting complex relationships between different task types.
-
Task Descriptions: The detailed task descriptions provided in ITDR were crucial for improving model performance in most subtasks, helping LLMs better comprehend the task requirements. However, for some generative tasks like Interest Recognition, task descriptions sometimes introduced noise, indicating a need for careful design in such cases.
-
Data Scale: Generally, more training data led to better performance, especially for user-item interaction tasks. User-item understanding tasks, however, were less sensitive to the amount of data.
-
Comparison with Larger Models: Remarkably, the open-source LLMs fine-tuned with ITDR outperformed larger, closed-source models like GPT-3.5-Turbo and Gemini 2.5 Flash on both user-item interaction and user-item understanding tasks. This highlights the significant value of specialized instruction tuning datasets like ITDR for recommendation systems.
This work marks a significant step in making LLMs more effective for recommendation systems by providing a much-needed, high-quality dataset. The researchers have made the ITDR dataset and the fine-tuned models publicly available to encourage further advancements in this area. You can find more details about this research paper here.
