Convolutional Feature Masking for Joint Object and Stuff Segmentation
This work addresses efficiency and accuracy issues in semantic segmentation for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the problem of semantic segmentation by proposing a method that masks convolutional features to exploit shape information, avoiding artificial boundaries and reducing computational cost. It achieves state-of-the-art results on PASCAL VOC and PASCAL-CONTEXT benchmarks with improved speed.
The topic of semantic segmentation has witnessed considerable progress due to the powerful features learned by convolutional neural networks (CNNs). The current leading approaches for semantic segmentation exploit shape information by extracting CNN features from masked image regions. This strategy introduces artificial boundaries on the images and may impact the quality of the extracted features. Besides, the operations on the raw image domain require to compute thousands of networks on a single image, which is time-consuming. In this paper, we propose to exploit shape information via masking convolutional features. The proposal segments (e.g., super-pixels) are treated as masks on the convolutional feature maps. The CNN features of segments are directly masked out from these maps and used to train classifiers for recognition. We further propose a joint method to handle objects and "stuff" (e.g., grass, sky, water) in the same framework. State-of-the-art results are demonstrated on benchmarks of PASCAL VOC and new PASCAL-CONTEXT, with a compelling computational speed.