Seeing What Is Not There: Learning Context to Determine Where Objects Are Missing
This work addresses a novel vision task for accessibility, specifically helping people with mobility disabilities by identifying missing curb ramps, though it is incremental in method.
The paper tackles the problem of identifying missing objects in images by training a standalone object-centric context representation to predict where objects should exist, even when absent. The approach is applied to finding missing curb ramps in city street images, showing effectiveness in this accessibility task.
Most of computer vision focuses on what is in an image. We propose to train a standalone object-centric context representation to perform the opposite task: seeing what is not there. Given an image, our context model can predict where objects should exist, even when no object instances are present. Combined with object detection results, we can perform a novel vision task: finding where objects are missing in an image. Our model is based on a convolutional neural network structure. With a specially designed training strategy, the model learns to ignore objects and focus on context only. It is fully convolutional thus highly efficient. Experiments show the effectiveness of the proposed approach in one important accessibility task: finding city street regions where curb ramps are missing, which could help millions of people with mobility disabilities.