KonIQ-10k: Towards an ecologically valid and large-scale IQA database
This addresses the problem of limited training data for deep learning methods in IQA, providing a scalable resource for researchers, though it is incremental as it builds on existing dataset creation approaches.
The authors tackled the lack of large-scale datasets for image quality assessment (IQA) by creating KonIQ-10k, a database of 10,073 images with 1.2 million quality ratings from 1,467 crowd workers, demonstrating its ecological validity through diversity analysis and comparisons to existing databases.
The main challenge in applying state-of-the-art deep learning methods to predict image quality in-the-wild is the relatively small size of existing quality scored datasets. The reason for the lack of larger datasets is the massive resources required in generating diverse and publishable content. We present a new systematic and scalable approach to create large-scale, authentic and diverse image datasets for Image Quality Assessment (IQA). We show how we built an IQA database, KonIQ-10k, consisting of 10,073 images, on which we performed very large scale crowdsourcing experiments in order to obtain reliable quality ratings from 1,467 crowd workers (1.2 million ratings). We argue for its ecological validity by analyzing the diversity of the dataset, by comparing it to state-of-the-art IQA databases, and by checking the reliability of our user studies.