A Next Basket Recommendation Reality Check
This work addresses evaluation issues in NBR for researchers and practitioners, providing insights into model design, but it is incremental as it focuses on analysis rather than a new recommendation method.
The authors tackled the evaluation problem in next basket recommendation (NBR) systems by introducing a novel distinction between repetition and exploration in predicted baskets, proposing new metrics to analyze state-of-the-art models and clarify the actual progress and reasons for improvements.
The goal of a next basket recommendation (NBR) system is to recommend items for the next basket for a user, based on the sequence of their prior baskets. Recently, a number of methods with complex modules have been proposed that claim state-of-the-art performance. They rarely look into the predicted basket and just provide intuitive reasons for the observed improvements, e.g., better representation, capturing intentions or relations, etc. We provide a novel angle on the evaluation of next basket recommendation methods, centered on the distinction between repetition and exploration: the next basket is typically composed of previously consumed items (i.e., repeat items) and new items (i.e, explore items). We propose a set of metrics that measure the repeat/explore ratio and performance of NBR models. Using these new metrics, we analyze state-of-the-art NBR models. The results of our analysis help to clarify the extent of the actual progress achieved by existing NBR methods as well as the underlying reasons for the improvements. Overall, our work sheds light on the evaluation problem of NBR and provides useful insights into the model design for this task.