Should we Embed? A Study on the Online Performance of Utilizing Embeddings for Real-Time Job Recommendations
This work addresses the challenge of improving recommendation performance for job seekers on a specific platform, but it is incremental as it builds on existing embedding techniques without introducing new paradigms.
The study tackled the problem of optimizing real-time job recommendations by evaluating embedding methods on an Austrian job platform, finding that using embeddings based on recent interactions improved Click-Through Rate for similar job recommendations, while combining frequency and recency embeddings enhanced personalization on the homepage.
In this work, we present the findings of an online study, where we explore the impact of utilizing embeddings to recommend job postings under real-time constraints. On the Austrian job platform Studo Jobs, we evaluate two popular recommendation scenarios: (i) providing similar jobs and, (ii) personalizing the job postings that are shown on the homepage. Our results show that for recommending similar jobs, we achieve the best online performance in terms of Click-Through Rate when we employ embeddings based on the most recent interaction. To personalize the job postings shown on a user's homepage, however, combining embeddings based on the frequency and recency with which a user interacts with job postings results in the best online performance.