Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature
This work addresses the need for multimedia systems to archive and retrieve large digitized art collections, but it is incremental as it focuses on optimizing existing methods for a specific domain.
The paper tackled the problem of measuring visual similarity between fine-art paintings to improve classification tasks like predicting style, genre, and artist, by investigating various visual features and metric learning approaches, resulting in an optimized similarity measure that enhanced prediction performance in experiments.
In the past few years, the number of fine-art collections that are digitized and publicly available has been growing rapidly. With the availability of such large collections of digitized artworks comes the need to develop multimedia systems to archive and retrieve this pool of data. Measuring the visual similarity between artistic items is an essential step for such multimedia systems, which can benefit more high-level multimedia tasks. In order to model this similarity between paintings, we should extract the appropriate visual features for paintings and find out the best approach to learn the similarity metric based on these features. We investigate a comprehensive list of visual features and metric learning approaches to learn an optimized similarity measure between paintings. We develop a machine that is able to make aesthetic-related semantic-level judgments, such as predicting a painting's style, genre, and artist, as well as providing similarity measures optimized based on the knowledge available in the domain of art historical interpretation. Our experiments show the value of using this similarity measure for the aforementioned prediction tasks.