Open Vocabulary Scene Parsing
This addresses the challenge of parsing scenes with a large and open vocabulary for computer vision applications, but it is incremental as it builds on existing embedding methods.
The paper tackles the problem of recognizing arbitrary objects in the wild by proposing a new task for open vocabulary scene parsing, using a joint image pixel and word concept embeddings framework, and validates it on the ADE20K dataset with results showing interpretability.
Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets. In this paper, we propose a new task that aims at parsing scenes with a large and open vocabulary, and several evaluation metrics are explored for this problem. Our proposed approach to this problem is a joint image pixel and word concept embeddings framework, where word concepts are connected by semantic relations. We validate the open vocabulary prediction ability of our framework on ADE20K dataset which covers a wide variety of scenes and objects. We further explore the trained joint embedding space to show its interpretability.