A backward pass through a CNN using a generative model of its activations
This addresses the limitation of CNNs in handling diverse queries beyond classification, offering a method for top-down and bottom-up fusion, though it appears incremental as it builds on existing backward pass concepts.
The paper tackled the problem of enabling CNNs to handle queries like object detection, reconstruction, and occlusion recovery by developing a backward pass through the network using a generative model of activations, resulting in a unified framework for these tasks.
Neural networks have shown to be a practical way of building a very complex mapping between a pre-specified input space and output space. For example, a convolutional neural network (CNN) mapping an image into one of a thousand object labels is approaching human performance in this particular task. However the mapping (neural network) does not automatically lend itself to other forms of queries, for example, to detect/reconstruct object instances, to enforce top-down signal on ambiguous inputs, or to recover object instances from occlusion. One way to address these queries is a backward pass through the network that fuses top-down and bottom-up information. In this paper, we show a way of building such a backward pass by defining a generative model of the neural network's activations. Approximate inference of the model would naturally take the form of a backward pass through the CNN layers, and it addresses the aforementioned queries in a unified framework.