Complex Relations in a Deep Structured Prediction Model for Fine Image Segmentation
This work addresses fine-grained segmentation for computer vision applications, but it is incremental as it builds on existing FCN-CRF models with additional relations.
The paper tackles the problem of poor segmentation performance on object parts in semantic segmentation models by incorporating containment and attachment relations as higher-order potentials in a CRF framework, showing positive effects on the Pascal VOC Parts dataset.
Many deep learning architectures for semantic segmentation involve a Fully Convolutional Neural Network (FCN) followed by a Conditional Random Field (CRF) to carry out inference over an image. These models typically involve unary potentials based on local appearance features computed by FCNs, and binary potentials based on the displacement between pixels. We show that while current methods succeed in segmenting whole objects, they perform poorly in situations involving a large number of object parts. We therefore suggest incorporating into the inference algorithm additional higher-order potentials inspired by the way humans identify and localize parts. We incorporate two relations that were shown to be useful to human object identification - containment and attachment - into the energy term of the CRF and evaluate their performance on the Pascal VOC Parts dataset. Our experimental results show that the segmentation of fine parts is positively affected by the addition of these two relations, and that the segmentation of fine parts can be further influenced by complex structural features.