Data Representativity for Machine Learning and AI Systems
This work addresses the issue of ensuring appropriate inference from data for AI practitioners, but it is incremental as it builds on existing notions without introducing a new paradigm.
The paper tackles the problem of data representativity in AI systems by reviewing definitions and introducing three measurable concepts to evaluate data samples, demonstrating the contrast between coverage and distribution representativity on US Census data.
Data representativity is crucial when drawing inference from data through machine learning models. Scholars have increased focus on unraveling the bias and fairness in models, also in relation to inherent biases in the input data. However, limited work exists on the representativity of samples (datasets) for appropriate inference in AI systems. This paper reviews definitions and notions of a representative sample and surveys their use in scientific AI literature. We introduce three measurable concepts to help focus the notions and evaluate different data samples. Furthermore, we demonstrate that the contrast between a representative sample in the sense of coverage of the input space, versus a representative sample mimicking the distribution of the target population is of particular relevance when building AI systems. Through empirical demonstrations on US Census data, we evaluate the opposing inherent qualities of these concepts. Finally, we propose a framework of questions for creating and documenting data with data representativity in mind, as an addition to existing dataset documentation templates.