ISDA: Position-Aware Instance Segmentation with Deformable Attention
This addresses the problem of non-end-to-end trainability in instance segmentation for computer vision researchers, offering an incremental improvement over existing methods.
The paper tackles the lack of end-to-end trainability in instance segmentation by proposing ISDA, a method that predicts object masks using learned position-aware kernels and features via deformable attention, eliminating the need for non-maximum suppression. It outperforms Mask R-CNN by 2.6 points on MS-COCO and achieves leading performance.
Most instance segmentation models are not end-to-end trainable due to either the incorporation of proposal estimation (RPN) as a pre-processing or non-maximum suppression (NMS) as a post-processing. Here we propose a novel end-to-end instance segmentation method termed ISDA. It reshapes the task into predicting a set of object masks, which are generated via traditional convolution operation with learned position-aware kernels and features of objects. Such kernels and features are learned by leveraging a deformable attention network with multi-scale representation. Thanks to the introduced set-prediction mechanism, the proposed method is NMS-free. Empirically, ISDA outperforms Mask R-CNN (the strong baseline) by 2.6 points on MS-COCO, and achieves leading performance compared with recent models. Code will be available soon.