Probabilistic Pixel-Adaptive Refinement Networks
This addresses boundary artifacts in dense prediction tasks like optical flow and semantic segmentation, offering an incremental improvement over existing image-adaptive methods.
The paper tackled the loss of location information and boundary artifacts in encoder-decoder networks for dense prediction tasks by introducing probabilistic pixel-adaptive convolutions (PPACs), which leverage both image guidance and network confidence, resulting in a clear reduction in boundary artifacts and substantial accuracy improvements on benchmarks for optical flow and semantic segmentation.
Encoder-decoder networks have found widespread use in various dense prediction tasks. However, the strong reduction of spatial resolution in the encoder leads to a loss of location information as well as boundary artifacts. To address this, image-adaptive post-processing methods have shown beneficial by leveraging the high-resolution input image(s) as guidance data. We extend such approaches by considering an important orthogonal source of information: the network's confidence in its own predictions. We introduce probabilistic pixel-adaptive convolutions (PPACs), which not only depend on image guidance data for filtering, but also respect the reliability of per-pixel predictions. As such, PPACs allow for image-adaptive smoothing and simultaneously propagating pixels of high confidence into less reliable regions, while respecting object boundaries. We demonstrate their utility in refinement networks for optical flow and semantic segmentation, where PPACs lead to a clear reduction in boundary artifacts. Moreover, our proposed refinement step is able to substantially improve the accuracy on various widely used benchmarks.