No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
This highlights a critical representation problem for developing world applications, emphasizing the need for geo-diverse datasets to avoid biased machine learning systems.
The study analyzed geo-diversity in two large open image datasets, finding amerocentric and eurocentric biases, and showed that classifiers trained on these datasets perform poorly on images from underrepresented locales, with strong performance differences across regions.
Modern machine learning systems such as image classifiers rely heavily on large scale data sets for training. Such data sets are costly to create, thus in practice a small number of freely available, open source data sets are widely used. We suggest that examining the geo-diversity of open data sets is critical before adopting a data set for use cases in the developing world. We analyze two large, publicly available image data sets to assess geo-diversity and find that these data sets appear to exhibit an observable amerocentric and eurocentric representation bias. Further, we analyze classifiers trained on these data sets to assess the impact of these training distributions and find strong differences in the relative performance on images from different locales. These results emphasize the need to ensure geo-representation when constructing data sets for use in the developing world.