Adversarially Robust Frame Sampling with Bounded Irregularities
This addresses a security vulnerability in video analysis systems for applications relying on efficient frame sampling, though it is incremental as it builds on existing sampling strategies.
The paper tackles the problem of adversarial attacks on fixed-rate frame sampling in video analysis by proposing a provably robust sampling method that introduces bounded irregularities.
In recent years, video analysis tools for automatically extracting meaningful information from videos are widely studied and deployed. Because most of them use deep neural networks which are computationally expensive, feeding only a subset of video frames into such algorithms is desired. Sampling the frames with fixed rate is always attractive for its simplicity, representativeness, and interpretability. For example, a popular cloud video API generated video and shot labels by processing only the first frame of every second in a video. However, one can easily attack such strategies by placing chosen frames at the sampled locations. In this paper, we present an elegant solution to this sampling problem that is provably robust against adversarial attacks and introduces bounded irregularities as well.