Cooperative Retriever and Ranker in Deep Recommenders
This work addresses inefficiencies in two-stage recommender systems for web services, offering an incremental improvement over existing joint training methods.
The paper tackles the problem of poor collaboration between retrieval and ranking stages in deep recommender systems, proposing a cooperative approach to address limitations like item distribution shift and false negatives, resulting in improved recommendation performance.
Deep recommender systems (DRS) are intensively applied in modern web services. To deal with the massive web contents, DRS employs a two-stage workflow: retrieval and ranking, to generate its recommendation results. The retriever aims to select a small set of relevant candidates from the entire items with high efficiency; while the ranker, usually more precise but time-consuming, is supposed to further refine the best items from the retrieved candidates. Traditionally, the two components are trained either independently or within a simple cascading pipeline, which is prone to poor collaboration effect. Though some latest works suggested to train retriever and ranker jointly, there still exist many severe limitations: item distribution shift between training and inference, false negative, and misalignment of ranking order. As such, it remains to explore effective collaborations between retriever and ranker.