VOCALExplore: Pay-as-You-Go Video Data Exploration and Model Building [Technical Report]
This system addresses the challenge of efficient video data exploration and model building for users in domain-specific applications, representing an incremental improvement with optimizations for latency and feature selection.
The authors tackled the problem of building domain-specific models over video datasets by introducing VOCALExplore, a system that supports interactive labeling and model training, achieving close to optimal model quality with low latency (~1 second per iteration) and no expensive preprocessing.
We introduce VOCALExplore, a system designed to support users in building domain-specific models over video datasets. VOCALExplore supports interactive labeling sessions and trains models using user-supplied labels. VOCALExplore maximizes model quality by automatically deciding how to select samples based on observed skew in the collected labels. It also selects the optimal video representations to use when training models by casting feature selection as a rising bandit problem. Finally, VOCALExplore implements optimizations to achieve low latency without sacrificing model performance. We demonstrate that VOCALExplore achieves close to the best possible model quality given candidate acquisition functions and feature extractors, and it does so with low visible latency (~1 second per iteration) and no expensive preprocessing.