Modeling Visual Compatibility through Hierarchical Mid-level Elements
This work addresses visual compatibility modeling for applications like fashion or product recommendation, but it appears incremental as it builds on existing CNN-based methods.
The paper tackles the problem of modeling visual compatibility between objects by proposing a hierarchical method to discover mid-level elements, achieving strong performance in describing object classes and compatibility characteristics on the Amazon dataset.
In this paper we present a hierarchical method to discover mid-level elements with the objective of modeling visual compatibility between objects. At the base-level, our method identifies patterns of CNN activations with the aim of modeling different variations/styles in which objects of the classes of interest may occur. At the top-level, the proposed method discovers patterns of co-occurring activations of base-level elements that define visual compatibility between pairs of object classes. Experiments on the massive Amazon dataset show the strength of our method at describing object classes and the characteristics that drive the compatibility between them.