Mask-aware IoU for Anchor Assignment in Real-time Instance Segmentation
This work improves real-time instance segmentation for applications like autonomous driving or video analysis by providing a faster and more accurate method, though it is incremental as it builds on existing assigners like ATSS.
The paper tackles the problem of anchor assignment in real-time instance segmentation by introducing Mask-aware Intersection-over-Union (maIoU), which incorporates ground truth masks for more accurate supervision, resulting in a model that achieves 37.7 mask AP at 25 fps on COCO test-dev, outperforming baseline methods by up to 6 AP and reducing inference time by 25%.
This paper presents Mask-aware Intersection-over-Union (maIoU) for assigning anchor boxes as positives and negatives during training of instance segmentation methods. Unlike conventional IoU or its variants, which only considers the proximity of two boxes; maIoU consistently measures the proximity of an anchor box with not only a ground truth box but also its associated ground truth mask. Thus, additionally considering the mask, which, in fact, represents the shape of the object, maIoU enables a more accurate supervision during training. We present the effectiveness of maIoU on a state-of-the-art (SOTA) assigner, ATSS, by replacing IoU operation by our maIoU and training YOLACT, a SOTA real-time instance segmentation method. Using ATSS with maIoU consistently outperforms (i) ATSS with IoU by $\sim 1$ mask AP, (ii) baseline YOLACT with fixed IoU threshold assigner by $\sim 2$ mask AP over different image sizes and (iii) decreases the inference time by $25 \%$ owing to using less anchors. Then, exploiting this efficiency, we devise maYOLACT, a faster and $+6$ AP more accurate detector than YOLACT. Our best model achieves $37.7$ mask AP at $25$ fps on COCO test-dev establishing a new state-of-the-art for real-time instance segmentation. Code is available at https://github.com/kemaloksuz/Mask-aware-IoU