Dense Semantic Correspondence where Every Pixel is a Classifier
This addresses a difficult computer vision problem for tasks requiring matching high-level structures, but it is incremental as it builds on existing detection and graphical model techniques.
The paper tackles the problem of dense semantic correspondence across objects and scenes with differing appearances and geometries by treating it as a constrained detection problem, where each pixel uses an exemplar LDA classifier, resulting in higher average precision and faster training compared to methods like exemplar SVM.
Determining dense semantic correspondences across objects and scenes is a difficult problem that underpins many higher-level computer vision algorithms. Unlike canonical dense correspondence problems which consider images that are spatially or temporally adjacent, semantic correspondence is characterized by images that share similar high-level structures whose exact appearance and geometry may differ. Motivated by object recognition literature and recent work on rapidly estimating linear classifiers, we treat semantic correspondence as a constrained detection problem, where an exemplar LDA classifier is learned for each pixel. LDA classifiers have two distinct benefits: (i) they exhibit higher average precision than similarity metrics typically used in correspondence problems, and (ii) unlike exemplar SVM, can output globally interpretable posterior probabilities without calibration, whilst also being significantly faster to train. We pose the correspondence problem as a graphical model, where the unary potentials are computed via convolution with the set of exemplar classifiers, and the joint potentials enforce smoothly varying correspondence assignment.