IVOA: Introspective Vision for Obstacle Avoidance
This work addresses the challenge of robust obstacle avoidance for autonomous mobile robots, but it is incremental as it builds on existing vision and sensor fusion methods.
The paper tackles the problem of unreliable vision-based perception for obstacle avoidance in autonomous robots by using a supervisory sensor to detect failures and generate a training dataset of reliable and unreliable image patches, resulting in a model that accurately predicts failures for two distinct vision algorithms in real-world indoor and outdoor environments.
Vision, as an inexpensive yet information rich sensor, is commonly used for perception on autonomous mobile robots. Unfortunately, accurate vision-based perception requires a number of assumptions about the environment to hold -- some examples of such assumptions, depending on the perception algorithm at hand, include purely lambertian surfaces, texture-rich scenes, absence of aliasing features, and refractive surfaces. In this paper, we present an approach for introspective vision for obstacle avoidance (IVOA) -- by leveraging a supervisory sensor that is occasionally available, we detect failures of stereo vision-based perception from divergence in plans generated by vision and the supervisory sensor. By projecting the 3D coordinates where the plans agree and disagree onto the images used for vision-based perception, IVOA generates a training set of reliable and unreliable image patches for perception. We then use this training dataset to learn a model of which image patches are likely to cause failures of the vision-based perception algorithm. Using this model, IVOA is then able to predict whether the relevant image patches in the observed images are likely to cause failures due to vision (both false positives and false negatives). We empirically demonstrate with extensive real-world data from both indoor and outdoor environments, the ability of IVOA to accurately predict the failures of two distinct vision algorithms.