AdaptIS: Adaptive Instance Selection Network
This addresses instance segmentation for complex or occluded objects, with incremental improvements in panoptic segmentation benchmarks.
The paper tackles class-agnostic instance segmentation by introducing AdaptIS, a network that generates masks for objects based on input points, achieving state-of-the-art results on Cityscapes and Mapillary benchmarks without COCO pretraining.
We present Adaptive Instance Selection network architecture for class-agnostic instance segmentation. Given an input image and a point $(x, y)$, it generates a mask for the object located at $(x, y)$. The network adapts to the input point with a help of AdaIN layers, thus producing different masks for different objects on the same image. AdaptIS generates pixel-accurate object masks, therefore it accurately segments objects of complex shape or severely occluded ones. AdaptIS can be easily combined with standard semantic segmentation pipeline to perform panoptic segmentation. To illustrate the idea, we perform experiments on a challenging toy problem with difficult occlusions. Then we extensively evaluate the method on panoptic segmentation benchmarks. We obtain state-of-the-art results on Cityscapes and Mapillary even without pretraining on COCO, and show competitive results on a challenging COCO dataset. The source code of the method and the trained models are available at https://github.com/saic-vul/adaptis.