A Hierarchical Grocery Store Image Dataset with Visual and Semantic Labels
This dataset addresses the problem of accurate image classification for assistive technology in daily activities like shopping for visually impaired individuals, but it is incremental as it focuses on a specific domain with new data.
The authors introduced a new benchmark dataset for classifying grocery store items like fruits, vegetables, and refrigerated products, aimed at assisting visually impaired people, and provided benchmark results using pretrained CNNs and a multi-view variational autoencoder.
Image classification models built into visual support systems and other assistive devices need to provide accurate predictions about their environment. We focus on an application of assistive technology for people with visual impairments, for daily activities such as shopping or cooking. In this paper, we provide a new benchmark dataset for a challenging task in this application - classification of fruits, vegetables, and refrigerated products, e.g. milk packages and juice cartons, in grocery stores. To enable the learning process to utilize multiple sources of structured information, this dataset not only contains a large volume of natural images but also includes the corresponding information of the product from an online shopping website. Such information encompasses the hierarchical structure of the object classes, as well as an iconic image of each type of object. This dataset can be used to train and evaluate image classification models for helping visually impaired people in natural environments. Additionally, we provide benchmark results evaluated on pretrained convolutional neural networks often used for image understanding purposes, and also a multi-view variational autoencoder, which is capable of utilizing the rich product information in the dataset.