CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios
This work addresses the problem of improving video search accuracy in real-world Chinese short video platforms, though it is incremental as it builds on existing vision-language models for a specific domain.
The authors tackled the lack of large-scale datasets for Chinese short video search by creating CBVS, a benchmark with millions of video covers and a manually labeled test set, and proposed UniCLIP, a method that integrates cover text semantics during training without relying on it at inference, achieving significant gains in deployment with hundreds of millions of visits.
Vision-Language Models pre-trained on large-scale image-text datasets have shown superior performance in downstream tasks such as image retrieval. Most of the images for pre-training are presented in the form of open domain common-sense visual elements. Differently, video covers in short video search scenarios are presented as user-originated contents that provide important visual summaries of videos. In addition, a portion of the video covers come with manually designed cover texts that provide semantic complements. In order to fill in the gaps in short video cover data, we establish the first large-scale cover-text benchmark for Chinese short video search scenarios. Specifically, we release two large-scale datasets CBVS-5M/10M to provide short video covers, and the manual fine-labeling dataset CBVS-20K to provide real user queries, which serves as an image-text benchmark test in the Chinese short video search field. To integrate the semantics of cover text in the case of modality missing, we propose UniCLIP where cover texts play a guiding role during training, however are not relied upon by inference. Extensive evaluation on CBVS-20K demonstrates the excellent performance of our proposal. UniCLIP has been deployed to Tencent's online video search systems with hundreds of millions of visits and achieved significant gains. The dataset and code are available at https://github.com/QQBrowserVideoSearch/CBVS-UniCLIP.