A Study on Fairness and Trust Perceptions in Automated Decision Making
This addresses fairness and trust issues in automated decision-making for users and stakeholders, but it is incremental as it builds on existing explanation methods.
The study investigated how different explanation methods for automated decision systems affect people's perceptions of fairness and trust, with a pilot study showing preliminary significant effects.
Automated decision systems are increasingly used for consequential decision making -- for a variety of reasons. These systems often rely on sophisticated yet opaque models, which do not (or hardly) allow for understanding how or why a given decision was arrived at. This is not only problematic from a legal perspective, but non-transparent systems are also prone to yield undesirable (e.g., unfair) outcomes because their sanity is difficult to assess and calibrate in the first place. In this work, we conduct a study to evaluate different attempts of explaining such systems with respect to their effect on people's perceptions of fairness and trustworthiness towards the underlying mechanisms. A pilot study revealed surprising qualitative insights as well as preliminary significant effects, which will have to be verified, extended and thoroughly discussed in the larger main study.