Anomalies by Synthesis: Anomaly Detection using Generative Diffusion Models for Off-Road Navigation
This addresses the challenge of safe navigation in unstructured environments for robots, though it is incremental as it builds on existing diffusion and vision-language models.
The paper tackles the problem of detecting out-of-distribution anomalies for off-road robot navigation by using a generative diffusion model to synthesize edited images and analyzing modifications, achieving accurate pixel-wise detection without assumptions about anomaly types.
In order to navigate safely and reliably in off-road and unstructured environments, robots must detect anomalies that are out-of-distribution (OOD) with respect to the training data. We present an analysis-by-synthesis approach for pixel-wise anomaly detection without making any assumptions about the nature of OOD data. Given an input image, we use a generative diffusion model to synthesize an edited image that removes anomalies while keeping the remaining image unchanged. Then, we formulate anomaly detection as analyzing which image segments were modified by the diffusion model. We propose a novel inference approach for guided diffusion by analyzing the ideal guidance gradient and deriving a principled approximation that bootstraps the diffusion model to predict guidance gradients. Our editing technique is purely test-time that can be integrated into existing workflows without the need for retraining or fine-tuning. Finally, we use a combination of vision-language foundation models to compare pixels in a learned feature space and detect semantically meaningful edits, enabling accurate anomaly detection for off-road navigation. Project website: https://siddancha.github.io/anomalies-by-diffusion-synthesis/