Multi-Class Zero-Shot Learning for Artistic Material Recognition
This work addresses material recognition in art for museums or researchers, but it is incremental as it applies an existing ZSL approach to a new domain.
The paper tackled the problem of identifying artistic materials in artworks using zero-shot learning, achieving a classification accuracy of 48.42% on 5,000 artworks from the Tate collection by learning from descriptions and materials in a separate dataset.
Zero-Shot Learning (ZSL) is an extreme form of transfer learning, where no labelled examples of the data to be classified are provided during the training stage. Instead, ZSL uses additional information learned about the domain, and relies upon transfer learning algorithms to infer knowledge about the missing instances. ZSL approaches are an attractive solution for sparse datasets. Here we outline a model to identify the materials with which a work of art was created, by learning the relationship between English descriptions of the subject of a piece and its composite materials. After experimenting with a range of hyper-parameters, we produce a model which is capable of correctly identifying the materials used on pieces from an entirely distinct museum dataset. This model returned a classification accuracy of 48.42% on 5,000 artworks taken from the Tate collection, which is distinct from the Rijksmuseum network used to create and train our model.