Detection and Description of Change in Visual Streams
This work addresses the challenge of analyzing changes in visual sequences for applications like surveillance or monitoring, representing an incremental advance by integrating language into existing detection methods.
The paper tackles the problem of detecting and describing changes in visual streams by introducing a framework that uses natural language descriptions to improve change detection accuracy, achieving significant improvements over methods that do not use language.
This paper presents a framework for the analysis of changes in visual streams: ordered sequences of images, possibly separated by significant time gaps. We propose a new approach to incorporating unlabeled data into training to generate natural language descriptions of change. We also develop a framework for estimating the time of change in visual stream. We use learned representations for change evidence and consistency of perceived change, and combine these in a regularized graph cut based change detector. Experimental evaluation on visual stream datasets, which we release as part of our contribution, shows that representation learning driven by natural language descriptions significantly improves change detection accuracy, compared to methods that do not rely on language.