RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems
This toolkit addresses the need for standardized robustness evaluation in recommender systems, which is incremental as it builds on existing robustness studies but provides a more unified approach.
The authors tackled the lack of comprehensive robustness evaluation tools for recommender systems by proposing a holistic view covering multiple dimensions like sub-populations and attacks, and they introduced RGRecSys, a toolkit that enables quick and uniform robustness assessment.
Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data. Due to the pervasiveness of recommender systems in online technologies, researchers have carried out several robustness studies focusing on data sparsity and profile injection attacks. Instead, we propose a more holistic view of robustness for recommender systems that encompasses multiple dimensions - robustness with respect to sub-populations, transformations, distributional disparity, attack, and data sparsity. While there are several libraries that allow users to compare different recommender system models, there is no software library for comprehensive robustness evaluation of recommender system models under different scenarios. As our main contribution, we present a robustness evaluation toolkit, Robustness Gym for RecSys (RGRecSys -- https://www.github.com/salesforce/RGRecSys), that allows us to quickly and uniformly evaluate the robustness of recommender system models.