TLDR: TalkPlayData 2 is a new synthetic dataset for multimodal conversational music recommendation. It uses an agentic pipeline where multiple specialized Large Language Models (LLMs) interact as a listener, a recommender, a profile generator, and a goal setter. This approach generates realistic, diverse, and multimodal conversations about music, incorporating audio, images, and text. Evaluations show it outperforms other datasets in relevance and naturalness, making it a valuable resource for training generative recommendation models.
In the rapidly evolving landscape of artificial intelligence, creating realistic and diverse datasets is crucial for training advanced models. A new research paper introduces TalkPlayData 2, an innovative synthetic dataset designed to power multimodal conversational music recommendation systems. This dataset is generated through a sophisticated pipeline involving multiple Large Language Model (LLM) agents, each playing a specialized role to simulate natural and coherent music discovery conversations.
The core idea behind TalkPlayData 2 is to mimic real-world interactions between a music listener and a recommendation system. Instead of relying on a single LLM, the pipeline employs an “agentic” approach, where four distinct LLMs collaborate: a Listener Profile LLM, a Conversation Goal LLM, a Listener LLM, and a Recsys LLM. This multi-agent design is a significant advancement, as it prevents the LLMs from “cheating” by accessing information not available to their real-world counterparts, thereby ensuring a high degree of realism in the generated data.
Each LLM agent is assigned a specific role with tailored prompts and access to different information subsets. For instance, the Listener Profile LLM analyzes a user’s listening history and demographic data to infer musical preferences, such as preferred culture, top artist, and genre. The Conversation Goal LLM then sets a session-level objective for the listener, ensuring a wide range of conversation scenarios, from discovering “songs about love” to finding “music from the 80s.” These goals also dictate the conversation’s expected length and complexity.
The actual conversation generation involves the Listener LLM and the Recsys LLM. The Listener LLM initiates the dialogue based on its profile and goal, while the Recsys LLM, equipped with the listener’s profile and a pool of music, responds with recommendations. A key aspect of TalkPlayData 2 is its multimodality. All LLMs are designed to understand and process audio and image data, alongside text. This means recommendations and queries can be based on various musical aspects, including audio snippets, album art, and lyrics, simulating a truly rich interaction.
To ensure diversity and prevent repetitive conversation styles, the system leverages user demographic information and pre-defined conversation goals. The base music data for the recommendations comes from the LFM-2b dataset, enriched with multimodal information like Spotify track identifiers, preview audio, album art, and estimated musical attributes like tempo, key, and chords. The Google Gemini 2.5 Flash LLM is used for the data generation process.
The quality of TalkPlayData 2 was rigorously evaluated using both an LLM-as-a-judge framework (employing the more advanced Gemini 2.5 Pro) and human evaluation. The results were highly promising. The LLM-as-a-judge evaluation confirmed the plausibility of conversation goals, the appropriateness of listener profiles, and the high quality of chat elements, including linguistic quality, helpfulness towards goals, and recommendation relevance. Human evaluators also rated TalkPlayData 2 as superior in both relevance and naturalness compared to existing conversational music recommendation datasets, including human-generated conversations and those created by single-LLM approaches.
Also Read:
- Advancing User Behavior Simulation with Persona-Driven Small Language Model Fine-Tuning
- VStyle: A New Benchmark for Teaching AI to Speak with Style and Emotion
This groundbreaking work, detailed in the paper TALK PLAYDATA 2: AN AGENTIC SYNTHETIC DATA PIPELINE FOR MULTIMODAL CONVERSATIONAL MUSIC RECOMMENDATION, represents a significant step forward in creating realistic training data for generative recommendation models. While the dataset already offers highly natural conversations, future directions include expanding the recommendation pool size, diversifying speaking styles, and incorporating longer audio snippets and more varied visual information to enhance its multimodal depth.
