SniffyArt: The Dataset of Smelling Persons
This work addresses the challenge of analyzing olfactory dimensions in visual arts for researchers in computer vision and art history, but it is incremental as it primarily provides a new dataset.
The paper tackles the problem of automated smell gesture recognition in historical artworks by introducing the SniffyArt dataset, which includes 1941 individuals from 441 artworks with annotations for bounding boxes, pose keypoints, and gesture labels, and presents baseline algorithm evaluations to showcase the potential of combining keypoint estimation with classification.
Smell gestures play a crucial role in the investigation of past smells in the visual arts yet their automated recognition poses significant challenges. This paper introduces the SniffyArt dataset, consisting of 1941 individuals represented in 441 historical artworks. Each person is annotated with a tightly fitting bounding box, 17 pose keypoints, and a gesture label. By integrating these annotations, the dataset enables the development of hybrid classification approaches for smell gesture recognition. The datasets high-quality human pose estimation keypoints are achieved through the merging of five separate sets of keypoint annotations per person. The paper also presents a baseline analysis, evaluating the performance of representative algorithms for detection, keypoint estimation, and classification tasks, showcasing the potential of combining keypoint estimation with smell gesture classification. The SniffyArt dataset lays a solid foundation for future research and the exploration of multi-task approaches leveraging pose keypoints and person boxes to advance human gesture and olfactory dimension analysis in historical artworks.