Generating Superpixels for High-resolution Images with Decoupled Patch Calibration
This work addresses a domain-specific problem for researchers and practitioners in computer vision who need efficient superpixel segmentation for high-resolution images, representing an incremental advancement by extending deep learning methods to higher resolutions.
The paper tackles the challenge of superpixel segmentation for high-resolution images, which is limited by memory and computation costs, by proposing Patch Calibration Networks (PCNet) with a Decoupled Patch Calibration branch. The result is a method that improves the resolution upper bound from 3K to 5K on 1080Ti GPUs and performs favorably against state-of-the-art methods in quantitative evaluations.
Superpixel segmentation has recently seen important progress benefiting from the advances in differentiable deep learning. However, the very high-resolution superpixel segmentation still remains challenging due to the expensive memory and computation cost, making the current advanced superpixel networks fail to process. In this paper, we devise Patch Calibration Networks (PCNet), aiming to efficiently and accurately implement high-resolution superpixel segmentation. PCNet follows the principle of producing high-resolution output from low-resolution input for saving GPU memory and relieving computation cost. To recall the fine details destroyed by the down-sampling operation, we propose a novel Decoupled Patch Calibration (DPC) branch for collaboratively augment the main superpixel generation branch. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. By sharing the parameters of DPC and main branches, the fine-detailed knowledge learned from high-resolution patches will be transferred to help calibrate the destroyed information. To the best of our knowledge, we make the first attempt to consider the deep-learning-based superpixel generation for high-resolution cases. To facilitate this research, we build evaluation benchmarks from two public datasets and one new constructed one, covering a wide range of diversities from fine-grained human parts to cityscapes. Extensive experiments demonstrate that our PCNet can not only perform favorably against the state-of-the-arts in the quantitative results but also improve the resolution upper bound from 3K to 5K on 1080Ti GPUs.