Unified Semantic Modeling Framework for Large-Scale Job Understanding at LinkedIn
This work addresses the challenge of scalable, cost-efficient job understanding for LinkedIn's talent-matching products, offering a practical solution for industry-scale text understanding systems.
LinkedIn developed a unified semantic modeling framework using a small language model (SLM) to transform noisy job postings into standardized attributes, achieving significant performance improvements and reduced operational complexity in offline and online A/B tests.
Job understanding is critical to LinkedIn's mission of connecting talent with opportunity. This task involves transforming unstructured and noisy job postings into standardized or derived job attributes that power numerous LinkedIn products. However, building a scalable, cost-efficient, and high-performing job understanding system remains challenging. In this paper, we present a unified semantic modeling framework powered by a small language model (SLM) to address the challenges. We begin by fine-tuning an open-source SLM using a suite of carefully curated synthetic tasks augmented with reasoning traces. These tasks jointly target taxonomy-guided classification and taxonomy-agnostic entity extraction. This allows the resulting model to acquire robust zero-shot generalization for job understanding in structured and unstructured contexts. Building upon this foundation, we introduce a multi-adapter architecture with attribute grouping to facilitate efficient task-specific adaptation while streamlining model management across diverse downstream attributes. Offline evaluations and online A/B tests demonstrate significant performance improvement while reducing operational complexity. Our work provides practical insights into building industry-scale text understanding systems.