Parametric Image Segmentation of Humans with Structural Shape Priors
This work addresses the challenging task of segmenting humans in complex, real-world images, which is important for applications like computer vision and robotics, but it is incremental as it builds on existing parametric max-flow and shape prior methods.
The paper tackles the problem of figure-ground segmentation of humans in natural images by proposing class-specific segmentation models that combine structural constraints and data-driven shape priors, achieving state-of-the-art results with a 20% improvement in hypothesis estimates and up to an order of magnitude smaller hypothesis set sizes on H3D and MPII datasets.
The figure-ground segmentation of humans in images captured in natural environments is an outstanding open problem due to the presence of complex backgrounds, articulation, varying body proportions, partial views and viewpoint changes. In this work we propose class-specific segmentation models that leverage parametric max-flow image segmentation and a large dataset of human shapes. Our contributions are as follows: (1) formulation of a sub-modular energy model that combines class-specific structural constraints and data-driven shape priors, within a parametric max-flow optimization methodology that systematically computes all breakpoints of the model in polynomial time; (2) design of a data-driven class-specific fusion methodology, based on matching against a large training set of exemplar human shapes (100,000 in our experiments), that allows the shape prior to be constructed on-the-fly, for arbitrary viewpoints and partial views. (3) demonstration of state of the art results, in two challenging datasets, H3D and MPII (where figure-ground segmentation annotations have been added by us), where we substantially improve on the first ranked hypothesis estimates of mid-level segmentation methods, by 20%, with hypothesis set sizes that are up to one order of magnitude smaller.