Designing Evaluations of Machine Learning Models for Subjective Inference: The Case of Sentence Toxicity
This work addresses bias in ML systems for subjective tasks, which is crucial for fairness in real-world applications, but it is incremental as it focuses on evaluation standards rather than novel methods.
The paper tackles the problem of bias in machine learning models applied to subjective inference, such as sentence toxicity detection, by proposing specifications for creating evaluation datasets that account for subjectivity and bias.
Machine Learning (ML) is increasingly applied in real-life scenarios, raising concerns about bias in automatic decision making. We focus on bias as a notion of opinion exclusion, that stems from the direct application of traditional ML pipelines to infer subjective properties. We argue that such ML systems should be evaluated with subjectivity and bias in mind. Considering the lack of evaluation standards yet to create evaluation benchmarks, we propose an initial list of specifications to define prior to creating evaluation datasets, in order to later accurately evaluate the biases. With the example of a sentence toxicity inference system, we illustrate how the specifications support the analysis of biases related to subjectivity. We highlight difficulties in instantiating these specifications and list future work for the crowdsourcing community to help the creation of appropriate evaluation datasets.