Zeyu Gong

Jing Tang, Quanlu Jia, Yuqiang Xie et al.

Generating high-quality shooting scripts containing information such as scene and shot language is essential for short drama script generation. We collect 6,660 popular short drama episodes from the Internet, each with an average of 100 short episodes, and the total number of short episodes is about 80,000, with a total duration of about 2,000 hours and totaling 10 terabytes (TB). We perform keyframe extraction and annotation on each episode to obtain about 10,000,000 shooting scripts. We perform 100 script restorations on the extracted shooting scripts based on our self-developed large short drama generation model SkyReels. This leads to a dataset containing 1,000,000,000 pairs of scripts and shooting scripts for short dramas, called SkyScript-100M. We compare SkyScript-100M with the existing dataset in detail and demonstrate some deeper insights that can be achieved based on SkyScript-100M. Based on SkyScript-100M, researchers can achieve several deeper and more far-reaching script optimization goals, which may drive a paradigm shift in the entire field of text-to-video and significantly advance the field of short drama video generation. The data and code are available at https://github.com/vaew/SkyScript-100M.

Jing Tang, Bo Tao, Zeyu Gong et al.

In recent years, multi-scale generative adversarial networks (GANs) have been proposed to build generalized image processing models based on single sample. Constraining on the sample size, multi-scale GANs have much difficulty converging to the global optimum, which ultimately leads to limitations in their capabilities. In this paper, we pioneered the introduction of PAC-Bayes generalized bound theory into the training analysis of specific models under different adversarial training methods, which can obtain a non-vacuous upper bound on the generalization error for the specified multi-scale GAN structure. Based on the drastic changes we found of the generalization error bound under different adversarial attacks and different training states, we proposed an adaptive training method which can greatly improve the image manipulation ability of multi-scale GANs. The final experimental results show that our adaptive training method in this paper has greatly contributed to the improvement of the quality of the images generated by multi-scale GANs on several image manipulation tasks. In particular, for the image super-resolution restoration task, the multi-scale GAN model trained by the proposed method achieves a 100% reduction in natural image quality evaluator (NIQE) and a 60% reduction in root mean squared error (RMSE), which is better than many models trained on large-scale datasets.

Yong Lee, Fuqiang Gu, Zeyu Gong et al.

This paper presents a novel surrogate-based cross-correlation (SBCC) framework to improve the correlation performance for practical particle image velocimetry~(PIV). The basic idea is that an optimized surrogate filter/image, replacing one raw image, will produce a more accurate and robust correlation signal. Specifically, the surrogate image is encouraged to generate perfect Gaussian-shaped correlation map to tracking particles (PIV image pair) while producing zero responses to image noise (context images). And the problem is formularized with an objective function composed of surrogate loss and consistency loss. As a result, the closed-form solution provides an efficient multivariate operator that could consider other negative context images. Compared with the state-of-the-art baseline methods (background subtraction, robust phase correlation, etc.), our SBCC method exhibits significant performance improvement (accuracy and robustness) on the synthetic dataset and several challenging experimental PIV cases. Besides, our implementation with experimental details (\url{https://github.com/yongleex/SBCC}) is also available for interested researchers.