TLDR: LANTERN is a novel AI framework developed by LinkedIn to improve job-person fit assessments and explanations. It addresses the challenges of deploying large language models (LLMs) for this domain-specific task by using a knowledge distillation approach. LANTERN trains smaller, efficient student models (one for classification and one for explanation) from a powerful teacher LLM, incorporating multi-level distillation and prompt engineering. This results in high-quality, interpretable outputs with low latency, leading to increased job seeker engagement and qualified applications in online evaluations.
Large language models (LLMs) have made significant strides in natural language processing, but deploying them for specialized tasks like assessing job-person fit on platforms such as LinkedIn presents unique hurdles. The core challenge lies in generating high-quality, actionable feedback—including both a fit assessment and a detailed explanation—while maintaining scalability and low inference latency for online use. Traditional LLMs are often too large and complex for such real-time, domain-specific applications.
To tackle these issues, LinkedIn researchers have introduced LANTERN, a new framework designed for scalable distillation of LLMs specifically for job-person fit and explanation tasks. LANTERN aims to transfer the extensive knowledge from a powerful, large teacher model into smaller, more efficient student models that are suitable for production environments.
Understanding LANTERN’s Approach
At its heart, LANTERN models the job-person fit task with multiple objectives, employing an encoder model for classification (determining the fit level) and a decoder model for generating explanations. The framework uses a multi-level knowledge distillation process, integrating both data-level and logit-level insights to effectively transfer knowledge from a robust black-box teacher model to these lightweight student models.
The process begins with an in-house teacher model, which is fine-tuned on a carefully curated dataset. This dataset is initially generated by prompting a strong black-box LLM (like GPT-4o) with job descriptions and candidate profiles. Human annotators then review and filter these outputs to ensure high quality, forming a ‘seed dataset’ for training the in-house teacher. This meticulous prompt engineering, which breaks down complex instructions into smaller subtasks, is crucial for achieving consistent and high-quality outputs from the teacher model.
Distilling Knowledge into Student Models
LANTERN then distills the knowledge from this in-house teacher into two distinct student models:
-
Explanation Model: This model is responsible for generating natural language rationales that justify the predicted match between a candidate and a job. It undergoes white-box distillation, learning to mimic the teacher’s output distribution to produce detailed explanations. This ensures high generation quality while maintaining inference efficiency.
-
Classification Model: This model predicts the fit level (e.g., Low, Medium, High) based on the job description and candidate profile. It uses an LLM as an encoder backbone, trained on a large, synthetically labeled corpus generated by the teacher model. This allows the classifier to generalize across a wide range of job-person fit scenarios.
Optimizations for Real-Time Serving
To meet the stringent real-time latency and high throughput requirements of an online platform, LANTERN incorporates several key optimizations:
-
Input Compression: Job descriptions are compressed by extracting only the most relevant requirement specifications, significantly reducing input length without compromising performance. This compression happens once during job creation and the summarized text is reused for subsequent inferences.
-
Model Size Reduction: The classification model is further downscaled to a very lightweight encoder (e.g., a 0.4B parameter ModernBERT variant) after input compression. This achieves comparable offline performance with a substantial improvement in serving throughput per GPU.
The online serving pipeline consists of a Summarization Module, a Classification Module, and an Explanation Module, each optimized for its specific traffic volume and latency requirements. For instance, the classification module handles 30 times more requests than the explanation module, necessitating its extreme efficiency.
Also Read:
- Advanced AI Framework Improves Student Learning Assessment in Digital Education
- Boosting AI Performance and Cutting Costs with MHA-RAG Soft Prompts
Impact and Future Implications
Extensive experimental results demonstrate that LANTERN significantly improves task-specific metrics for both job-person fit and explanation. Online evaluations further confirm its effectiveness, showing measurable gains in job seeker engagement, including a +0.24% increase in apply rate and a +0.28% increase in qualified applications. The framework has been successfully deployed in production at LinkedIn, powering job-person evaluation with tangible business impact.
This research highlights the importance of practical techniques like synthetic data generation, prompt engineering, and multi-level distillation for building efficient and interpretable LLM-based systems at scale. For more technical details, you can refer to the full research paper here.
