Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
This addresses a key limitation in compositional zero-shot learning for computer vision by enabling recognition without prior knowledge of test compositions, though it is incremental as it builds on existing graph-based approaches.
The paper tackles the problem of open-world compositional zero-shot learning, where unseen compositions are unlimited at test time, by proposing Co-CGE, a graph-based method that models dependencies and estimates feasibility scores, achieving state-of-the-art performance in standard settings and outperforming previous methods in open-world scenarios.
Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. A problem with standard CZSL is the assumption of knowing which unseen compositions will be available at test time. In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions. To address this problem, we propose a new approach, Compositional Cosine Graph Embeddings (Co-CGE), based on two principles. First, Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network. The graph propagates information from seen to unseen concepts, improving their representations. Second, since not all unseen compositions are equally feasible, and less feasible ones may damage the learned representations, Co-CGE estimates a feasibility score for each unseen composition, using the scores as margins in a cosine similarity-based loss and as weights in the adjacency matrix of the graphs. Experiments show that our approach achieves state-of-the-art performances in standard CZSL while outperforming previous methods in the open world scenario.