Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks
This addresses the problem of real-world adoption of NLP in HR applications by providing a comprehensive benchmark and efficient models, though it is incremental as it builds on existing distillation and graph-based methods.
The paper tackles the lack of benchmarks and smaller models for HR-related NLP tasks by introducing the Resume-Job Description Benchmark (RJDB), which includes over 50,000 triples, and shows that student models trained on it achieve near or better performance than GPT-4.
Numerous HR applications are centered around resumes and job descriptions. While they can benefit from advancements in NLP, particularly large language models, their real-world adoption faces challenges due to absence of comprehensive benchmarks for various HR tasks, and lack of smaller models with competitive capabilities. In this paper, we aim to bridge this gap by introducing the Resume-Job Description Benchmark (RJDB). We meticulously craft this benchmark to cater to a wide array of HR tasks, including matching and explaining resumes to job descriptions, extracting skills and experiences from resumes, and editing resumes. To create this benchmark, we propose to distill domain-specific knowledge from a large language model (LLM). We rely on a curated skill-occupation graph to ensure diversity and provide context for LLMs generation. Our benchmark includes over 50 thousand triples of job descriptions, matched resumes and unmatched resumes. Using RJDB, we train multiple smaller student models. Our experiments reveal that the student models achieve near/better performance than the teacher model (GPT-4), affirming the effectiveness of the benchmark. Additionally, we explore the utility of RJDB on out-of-distribution data for skill extraction and resume-job description matching, in zero-shot and weak supervision manner. We release our datasets and code to foster further research and industry applications.