SceneDiff: A Benchmark and Method for Multiview Object Change Detection
It addresses multiview object change detection for applications like robotic tidying and construction monitoring, with a novel benchmark and method.
The paper tackles the problem of identifying objects that have been added, removed, or moved between multiview captures of the same scene, introducing a new benchmark and a training-free method that achieves 94% and 37.4% relative AP improvements over existing approaches.
We investigate the problem of identifying objects that have been added, removed, or moved between a pair of captures (images or videos) of the same scene at different times. Detecting such changes is important for many applications, such as robotic tidying or construction progress and safety monitoring. A major challenge is that varying viewpoints can cause objects to falsely appear changed. We introduce SceneDiff Benchmark, the first multiview change detection benchmark with object instance annotations, comprising 350 diverse video pairs with thousands of changed objects. We also introduce the SceneDiff method, a new training-free approach for multiview object change detection that leverages pretrained 3D, segmentation, and image encoding models to robustly predict across multiple benchmarks. Our method aligns the captures in 3D, extracts object regions, and compares spatial and semantic region features to detect changes. Experiments on multi-view and two-view benchmarks demonstrate that our method outperforms existing approaches by large margins (94% and 37.4% relative AP improvements). The benchmark and code will be publicly released.