Unified Perceptual Parsing for Scene Understanding
This addresses the challenge of comprehensive scene understanding for machine vision systems, representing a novel task rather than an incremental improvement.
The paper tackles the problem of recognizing multiple visual concepts from a single image by introducing Unified Perceptual Parsing, developing UPerNet and a training strategy to learn from heterogeneous annotations, and shows it effectively segments a wide range of concepts.
Humans recognize the visual world at multiple levels: we effortlessly categorize scenes and detect objects inside, while also identifying the textures and surfaces of the objects along with their different compositional parts. In this paper, we study a new task called Unified Perceptual Parsing, which requires the machine vision systems to recognize as many visual concepts as possible from a given image. A multi-task framework called UPerNet and a training strategy are developed to learn from heterogeneous image annotations. We benchmark our framework on Unified Perceptual Parsing and show that it is able to effectively segment a wide range of concepts from images. The trained networks are further applied to discover visual knowledge in natural scenes. Models are available at \url{https://github.com/CSAILVision/unifiedparsing}.