ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentation
This work addresses the problem of deploying efficient panoptic segmentation models for real-world applications, though it is incremental as it builds on existing mask transformer architectures.
The paper tackles the difficulty of training efficient mask transformers for panoptic segmentation by introducing ReMaX, a relaxation mechanism during training that reduces false positive penalization, leading to state-of-the-art results on COCO, ADE20K, and Cityscapes without extra inference cost.
This paper presents a new mechanism to facilitate the training of mask transformers for efficient panoptic segmentation, democratizing its deployment. We observe that due to its high complexity, the training objective of panoptic segmentation will inevitably lead to much higher false positive penalization. Such unbalanced loss makes the training process of the end-to-end mask-transformer based architectures difficult, especially for efficient models. In this paper, we present ReMaX that adds relaxation to mask predictions and class predictions during training for panoptic segmentation. We demonstrate that via these simple relaxation techniques during training, our model can be consistently improved by a clear margin \textbf{without} any extra computational cost on inference. By combining our method with efficient backbones like MobileNetV3-Small, our method achieves new state-of-the-art results for efficient panoptic segmentation on COCO, ADE20K and Cityscapes. Code and pre-trained checkpoints will be available at \url{https://github.com/google-research/deeplab2}.