Appearance invariance in convolutional networks with neighborhood similarity
This addresses a key limitation in computer vision for applications requiring robustness to appearance variations, though it is an incremental improvement based on Gestalt principles.
The paper tackles the problem of convolutional networks' poor generalization to novel appearances not seen during training by introducing a neighborhood similarity layer (NSL) that induces appearance invariance, demonstrating its effectiveness in digit recognition, semantic labeling, and cell detection tasks.
We present a neighborhood similarity layer (NSL) which induces appearance invariance in a network when used in conjunction with convolutional layers. We are motivated by the observation that, even though convolutional networks have low generalization error, their generalization capability does not extend to samples which are not represented by the training data. For instance, while novel appearances of learned concepts pose no problem for the human visual system, feedforward convolutional networks are generally not successful in such situations. Motivated by the Gestalt principle of grouping with respect to similarity, the proposed NSL transforms its input feature map using the feature vectors at each pixel as a frame of reference, i.e. center of attention, for its surrounding neighborhood. This transformation is spatially varying, hence not a convolution. It is differentiable; therefore, networks including the proposed layer can be trained in an end-to-end manner. We analyze the invariance of NSL to significant changes in appearance that are not represented in the training data. We also demonstrate its advantages for digit recognition, semantic labeling and cell detection problems.