Use square root affinity to regress labels in semantic segmentation
This work addresses the challenge of semantic segmentation in computer vision, but it is incremental as it builds on existing affinity-based methods.
The paper tackles the problem of improving semantic segmentation by using affinity matrices in a supervised manner, resulting in a novel Affinity Regression loss that enhances network performance on NYUv2 and Cityscapes datasets.
Semantic segmentation is a basic but non-trivial task in computer vision. Many previous work focus on utilizing affinity patterns to enhance segmentation networks. Most of these studies use the affinity matrix as a kind of feature fusion weights, which is part of modules embedded in the network, such as attention models and non-local models. In this paper, we associate affinity matrix with labels, exploiting the affinity in a supervised way. Specifically, we utilize the label to generate a multi-scale label affinity matrix as a structural supervision, and we use a square root kernel to compute a non-local affinity matrix on output layers. With such two affinities, we define a novel loss called Affinity Regression loss (AR loss), which can be an auxiliary loss providing pair-wise similarity penalty. Our model is easy to train and adds little computational burden without run-time inference. Extensive experiments on NYUv2 dataset and Cityscapes dataset demonstrate that our proposed method is sufficient in promoting semantic segmentation networks.