RecNextEval: A Reference Implementation for Temporal Next-Batch Recommendation Evaluation
For recommender systems researchers, this tool addresses concerns about evaluation validity by providing a reference implementation that simulates production environments more accurately.
The paper presents RecNextEval, an open-source evaluation framework for next-batch recommendation that uses a time-window data split to minimize data leakage and improve evaluation validity.
A good number of toolkits have been developed in Recommender Systems (RecSys) research to promote fair evaluation and reproducibility. However, recent critical examinations of RecSys evaluation protocols have raised concerns regarding the validity of existing evaluation pipelines. In this demonstration, we present RecNextEval, a reference implementation of an evaluation framework specifically designed for next-batch recommendation. RecNextEval utilizes a time-window data split to ensure models are evaluated along a global timeline, effectively minimizing data leakage. Our implementation highlights the inherent complexities of RecSys evaluation and encourages a shift toward model development that more accurately simulates production environments. The RecNextEval library and its accompanying GUI interface are open-source and publicly accessible.