Toward a Taxonomy and Computational Models of Abnormalities in Images
This work addresses the challenge of understanding atypicalities in images for computer vision applications, representing an incremental advancement through a more comprehensive approach.
The paper tackles the problem of recognizing and categorizing abnormalities in images, which has been understudied in computer vision, by proposing a new dataset, a taxonomy based on human experiments, and a computational model that outperforms prior methods in abnormality recognition.
The human visual system can spot an abnormal image, and reason about what makes it strange. This task has not received enough attention in computer vision. In this paper we study various types of atypicalities in images in a more comprehensive way than has been done before. We propose a new dataset of abnormal images showing a wide range of atypicalities. We design human subject experiments to discover a coarse taxonomy of the reasons for abnormality. Our experiments reveal three major categories of abnormality: object-centric, scene-centric, and contextual. Based on this taxonomy, we propose a comprehensive computational model that can predict all different types of abnormality in images and outperform prior arts in abnormality recognition.