A Unified Causal Framework for Auditing Recommender Systems for Ethical Concerns
This work addresses ethical auditing for recommender systems, which is crucial for mitigating biases and ensuring user agency, though it is incremental as it builds on existing causal approaches.
The paper tackles the problem of auditing recommender systems for ethical concerns by proposing a unified causal framework, which identifies gaps in existing metrics and introduces new ones like future- and past-reachability and stability to measure user agency, with experiments demonstrating their efficacy.
As recommender systems become widely deployed in different domains, they increasingly influence their users' beliefs and preferences. Auditing recommender systems is crucial as it not only ensures the continuous improvement of recommendation algorithms but also safeguards against potential issues like biases and ethical concerns. In this paper, we view recommender system auditing from a causal lens and provide a general recipe for defining auditing metrics. Under this general causal auditing framework, we categorize existing auditing metrics and identify gaps in them -- notably, the lack of metrics for auditing user agency while accounting for the multi-step dynamics of the recommendation process. We leverage our framework and propose two classes of such metrics:future- and past-reacheability and stability, that measure the ability of a user to influence their own and other users' recommendations, respectively. We provide both a gradient-based and a black-box approach for computing these metrics, allowing the auditor to compute them under different levels of access to the recommender system. In our experiments, we demonstrate the efficacy of methods for computing the proposed metrics and inspect the design of recommender systems through these proposed metrics.