VC-Agent: An Interactive Agent for Customized Video Dataset Collection
This addresses the challenge of personalized video dataset collection for researchers and practitioners, offering a novel interactive approach that is incremental in leveraging existing models.
The paper tackles the labor-intensive problem of collecting customized video datasets by proposing VC-Agent, an interactive agent that uses user queries and feedback to retrieve and scale up relevant video clips with minimal input, demonstrating effectiveness and efficiency in real scenarios.
Facing scaling laws, video data from the internet becomes increasingly important. However, collecting extensive videos that meet specific needs is extremely labor-intensive and time-consuming. In this work, we study the way to expedite this collection process and propose VC-Agent, the first interactive agent that is able to understand users' queries and feedback, and accordingly retrieve/scale up relevant video clips with minimal user input. Specifically, considering the user interface, our agent defines various user-friendly ways for the user to specify requirements based on textual descriptions and confirmations. As for agent functions, we leverage existing multi-modal large language models to connect the user's requirements with the video content. More importantly, we propose two novel filtering policies that can be updated when user interaction is continually performed. Finally, we provide a new benchmark for personalized video dataset collection, and carefully conduct the user study to verify our agent's usage in various real scenarios. Extensive experiments demonstrate the effectiveness and efficiency of our agent for customized video dataset collection. Project page: https://allenyidan.github.io/vcagent_page/.