Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model
This addresses the problem of amodal segmentation for computer vision researchers, offering a novel approach that reduces the need for hard-to-obtain annotated data, though it is incremental in applying Bayesian methods to this specific domain.
The paper tackles amodal segmentation, a challenging computer vision task to segment occluded object boundaries, by formulating it as an out-of-task and out-of-distribution generalization problem using a Bayesian generative model. The result shows that the algorithm outperforms alternative methods with the same supervision by a large margin and even beats methods using annotated amodal segmentations during training when occlusion is high.
Amodal completion is a visual task that humans perform easily but which is difficult for computer vision algorithms. The aim is to segment those object boundaries which are occluded and hence invisible. This task is particularly challenging for deep neural networks because data is difficult to obtain and annotate. Therefore, we formulate amodal segmentation as an out-of-task and out-of-distribution generalization problem. Specifically, we replace the fully connected classifier in neural networks with a Bayesian generative model of the neural network features. The model is trained from non-occluded images using bounding box annotations and class labels only, but is applied to generalize out-of-task to object segmentation and to generalize out-of-distribution to segment occluded objects. We demonstrate how such Bayesian models can naturally generalize beyond the training task labels when they learn a prior that models the object's background context and shape. Moreover, by leveraging an outlier process, Bayesian models can further generalize out-of-distribution to segment partially occluded objects and to predict their amodal object boundaries. Our algorithm outperforms alternative methods that use the same supervision by a large margin, and even outperforms methods where annotated amodal segmentations are used during training, when the amount of occlusion is large. Code is publicly available at https://github.com/YihongSun/Bayesian-Amodal.