Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure
This addresses the issue of over-recommending popular items and under-exposing niche content for users in large-scale recommendation platforms, representing a new paradigm rather than an incremental improvement.
The paper tackled the problem of popularity bias in recommender systems by introducing an exposure-aware retrieval scoring approach, which increased uniquely retrieved items by 25% and reduced over-popular content dominance by 40% while maintaining user engagement.
Recommender systems operate in closed feedback loops, where user interactions reinforce popularity bias, leading to over-recommendation of already popular items while under-exposing niche or novel content. Existing bias mitigation methods, such as Inverse Propensity Scoring (IPS) and Off-Policy Correction (OPC), primarily operate at the ranking stage or during training, lacking explicit real-time control over exposure dynamics. In this work, we introduce an exposure-aware retrieval scoring approach, which explicitly models item exposure probability and adjusts retrieval-stage ranking at inference time. Unlike prior work, this method decouples exposure effects from engagement likelihood, enabling controlled trade-offs between fairness and engagement in large-scale recommendation platforms. We validate our approach through online A/B experiments in a real-world video recommendation system, demonstrating a 25% increase in uniquely retrieved items and a 40% reduction in the dominance of over-popular content, all while maintaining overall user engagement levels. Our results establish a scalable, deployable solution for mitigating popularity bias at the retrieval stage, offering a new paradigm for bias-aware personalization.