TLDR: LinkedIn has developed a new retrieval system for its Feed, fine-tuning Meta’s LLaMA 3 as a dual encoder to generate high-quality user and content embeddings from text. This approach simplifies architecture, improves relevance, and enhances efficiency. Key findings include the effectiveness of InfoNCE loss, hard negative mining, Matryoshka embeddings for efficiency, and quantizing numerical features in prompts. Online A/B tests showed significant improvements in member engagement and revenue, particularly for new members and those with fewer connections.
LinkedIn, a leading professional networking platform, constantly strives to enhance its user experience by delivering highly relevant content in its feed. This is a monumental task, considering the platform needs to sift through hundreds of millions of potential posts to select a mere 2000 candidates for a member’s feed, all within a few milliseconds. A recent research paper from LinkedIn details a groundbreaking approach to tackle this challenge: leveraging large causal language models for large-scale content retrieval.
The paper, titled Large Scale Retrieval for the LinkedIn Feed using Causal Language Models, introduces a novel retrieval system that fine-tunes Meta’s LLaMA 3, a powerful causal language model, to act as a ‘dual encoder’. This system generates high-quality numerical representations, known as embeddings, for both users (members) and content (items) using only textual input. This innovation aims to streamline and improve the efficiency of content recommendations, moving beyond the complex multi-index architectures previously in place.
The Challenge of Content Retrieval
Traditional recommendation systems, including LinkedIn’s, rely on various retrieval techniques like inverted indices, trending sources, and collaborative filtering. While effective, this multi-index approach can lead to significant engineering complexity and operational overhead. The advent of Large Language Models (LLMs) offers a compelling opportunity to simplify these architectures due to their strong capabilities in understanding and representing semantic information.
A Unified LLM-Based Approach
The core of LinkedIn’s new system is a fine-tuned LLaMA 3 model operating as a dual encoder. In this setup, the model processes member information and content information separately to create distinct embeddings. These embeddings are then compared using a similarity function, like cosine similarity, to identify the most relevant content for a given member.
Crafting the Input: Prompt Design
A crucial aspect of this system is how information is fed to the LLM. Both member and content features are converted into structured text prompts. For content, features include post type, shared media, author details, popularity, and the post’s text. For members, it includes their name, profile headline, industry, skills, and a history of posts they’ve positively engaged with. The research found that including only positive interactions in a member’s history significantly improved recall metrics.
Training for Relevance: Loss Functions and Negative Mining
To optimize the model, researchers experimented with different training objectives. The InfoNCE loss function proved more effective than Binary Cross-Entropy (BCE) in drawing positive member-item pairs closer in the embedding space while pushing negative pairs apart. Furthermore, the system incorporates ‘easy’ and ‘hard’ negative mining. Easy negatives are other items in the same training batch, while hard negatives are items a member saw but didn’t engage with. Adding these hard negatives significantly boosted the model’s recall performance.
Efficiency with Matryoshka Embeddings
To address computational and memory constraints in a production environment, the team employed Matryoshka Representation Learning (MRL). This technique allows the model to learn nested, size-adaptive representations. Essentially, it means that smaller subsets of the generated embeddings can still be highly effective for retrieval, offering flexibility in deployment without needing to retrain the model for different embedding sizes. The study showed that reducing embedding dimensions to 512 did not significantly impact recall, offering substantial storage cost reductions.
Capturing Numerical Features
A key learning was how to enable LLMs to effectively capture numerical features, such as post popularity counts, which are vital for the final ranking model. By ‘quantizing’ these count features (e.g., converting raw counts into percentages) and including them in the input prompt, the LLM was able to encode this information better into the embeddings, leading to improved correlation between retrieval scores and popularity, and a 15% improvement in recall.
Real-World Impact: Online A/B Testing
After extensive offline evaluation, the LLM-based retrieval system was A/B tested against LinkedIn’s existing suggested content model. The results were compelling:
- Revenue increased by +0.8%.
- Daily Unique Professional Interactors increased by +0.2%.
The impact was particularly significant for newer members or those with fewer network connections, who often lack strong engagement signals. For this group:
- Daily Active Unique Users increased by +0.23%.
- Daily Unique Professional Interactions increased by +1.17%.
- Revenue increased by +3.29%.
These results underscore that high-quality suggested content plays a crucial role in member retention and engagement, especially for those with less established networks.
Also Read:
- Unifying User Interests Across Diverse Online Scenarios with LLM-Enhanced Recommendations
- Enhancing Search-Augmented LLMs with Verifiable Nugget-Based Rewards
Looking Ahead
This work demonstrates the effective adaptation of generative language models for real-time, high-throughput retrieval in industrial applications. LinkedIn plans further improvements, including better handling of unimpressed content, exploring distillation strategies, further reducing embedding dimensionality, and investigating LLM-powered embeddings for content from members’ connections. They are also prototyping user-prompt-driven feed recommendations as a re-ranking layer, showing promising early results.
