Zhongwei Zhang

Zhongwei Zhang, Fuchen Long, Wei Li et al.

The In-context generation paradigm recently has demonstrated strong power in instructional image editing with both data efficiency and synthesis quality. Nevertheless, shaping such in-context learning for instruction-based video editing is not trivial. Without specifying editing regions, the results can suffer from the problem of inaccurate editing regions and the token interference between editing and non-editing areas during denoising. To address these, we present ReCo, a new instructional video editing paradigm that novelly delves into constraint modeling between editing and non-editing regions during in-context generation. Technically, ReCo width-wise concatenates source and target video for joint denoising. To calibrate video diffusion learning, ReCo capitalizes on two regularization terms, i.e., latent and attention regularization, conducting on one-step backward denoised latents and attention maps, respectively. The former increases the latent discrepancy of the editing region between source and target videos while reducing that of non-editing areas, emphasizing the modification on editing area and alleviating outside unexpected content generation. The latter suppresses the attention of tokens in the editing region to the tokens in counterpart of the source video, thereby mitigating their interference during novel object generation in target video. Furthermore, we propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training. Extensive experiments conducted on four major instruction-based video editing tasks demonstrate the superiority of our proposal.

Zhongwei Zhang, Fuchen Long, Yingwei Pan et al.

Recent advances in text-to-video generation have demonstrated the utility of powerful diffusion models. Nevertheless, the problem is not trivial when shaping diffusion models to animate static image (i.e., image-to-video generation). The difficulty originates from the aspect that the diffusion process of subsequent animated frames should not only preserve the faithful alignment with the given image but also pursue temporal coherence among adjacent frames. To alleviate this, we present TRIP, a new recipe of image-to-video diffusion paradigm that pivots on image noise prior derived from static image to jointly trigger inter-frame relational reasoning and ease the coherent temporal modeling via temporal residual learning. Technically, the image noise prior is first attained through one-step backward diffusion process based on both static image and noised video latent codes. Next, TRIP executes a residual-like dual-path scheme for noise prediction: 1) a shortcut path that directly takes image noise prior as the reference noise of each frame to amplify the alignment between the first frame and subsequent frames; 2) a residual path that employs 3D-UNet over noised video and static image latent codes to enable inter-frame relational reasoning, thereby easing the learning of the residual noise for each frame. Furthermore, both reference and residual noise of each frame are dynamically merged via attention mechanism for final video generation. Extensive experiments on WebVid-10M, DTDB and MSR-VTT datasets demonstrate the effectiveness of our TRIP for image-to-video generation. Please see our project page at https://trip-i2v.github.io/TRIP/.

Olivier C. Pasche, Jonathan Wider, Zhongwei Zhang et al.

The forecast accuracy of machine learning (ML) weather prediction models is improving rapidly, leading many to speak of a "second revolution in weather forecasting". With numerous methods being developed and limited physical guarantees offered by ML models, there is a critical need for a comprehensive evaluation of these emerging techniques. While this need has been partly fulfilled by benchmark datasets, they provide little information on rare and impactful extreme events or on compound impact metrics, for which model accuracy might degrade due to misrepresented dependencies between variables. To address these issues, we compare ML weather prediction models (GraphCast, PanguWeather, and FourCastNet) and ECMWF's high-resolution forecast system (HRES) in three case studies: the 2021 Pacific Northwest heatwave, the 2023 South Asian humid heatwave, and the North American winter storm in 2021. We find that ML weather prediction models locally achieve similar accuracy to HRES on the record-shattering Pacific Northwest heatwave but underperform when aggregated over space and time. However, they forecast the compound winter storm substantially better. We also highlight structural differences in how the errors of HRES and the ML models build up to that event. The ML forecasts lack important variables for a detailed assessment of the health risks of the 2023 humid heatwave. Using a possible substitute variable, prediction errors show spatial patterns with the highest danger levels over Bangladesh being underestimated by the ML models. Generally, case-study-driven, impact-centric evaluation can complement existing research, increase public trust, and aid in developing reliable ML weather prediction models.

Zhongwei Zhang, Erich Fischer, Jakob Zscheischler et al.

Artificial intelligence (AI)-based models are revolutionizing weather forecasting and have surpassed leading numerical weather prediction systems on various benchmark tasks. However, their ability to extrapolate and reliably forecast unprecedented extreme events remains unclear. Here, we show that for record-breaking weather extremes, the numerical model High RESolution forecast (HRES) from the European Centre for Medium-Range Weather Forecasts still consistently outperforms state-of-the-art AI models GraphCast, GraphCast operational, Pangu-Weather, Pangu-Weather operational, and Fuxi. We demonstrate that forecast errors in AI models are consistently larger for record-breaking heat, cold, and wind than in HRES across nearly all lead times. We further find that the examined AI models tend to underestimate both the frequency and intensity of record-breaking events, and they underpredict hot records and overestimate cold records with growing errors for larger record exceedance. Our findings underscore the current limitations of AI weather models in extrapolating beyond their training domain and in forecasting the potentially most impactful record-breaking weather events that are particularly frequent in a rapidly warming climate. Further rigorous verification and model development is needed before these models can be solely relied upon for high-stakes applications such as early warning systems and disaster management.

Mishall Al-Zubaidie, Zhongwei Zhang, Ji Zhang

Public-key cryptography algorithms, especially elliptic curve cryptography (ECC) and elliptic curve digital signature algorithm (ECDSA) have been attracting attention from many researchers in different institutions because these algorithms provide security and high performance when being used in many areas such as electronic-healthcare, electronic-banking, electronic-commerce, electronic-vehicular, and electronic-governance. These algorithms heighten security against various attacks and at the same time improve performance to obtain efficiencies (time, memory, reduced computation complexity, and energy saving) in an environment of the constrained source and large systems. This paper presents detailed and a comprehensive survey of an update of the ECDSA algorithm in terms of performance, security, and applications.

Mishall Al-Zubaidie, Zhongwei Zhang, Ji Zhang

Providing a mechanism to authenticate users in healthcare applications is an essential security requirement to prevent both external and internal attackers from penetrating patients' identities and revealing their health data. Many schemes have been developed to provide authentication mechanisms to ensure that only legitimate users are authorized to connect, but these schemes still suffer from vulnerable security. Various attacks expose patients' data for malicious tampering or destruction. Transferring health-related data and information between users and the health centre makes them exposed to penetration by adversaries as they may move through an insecure channel. In addition, previous mechanisms have suffered from the poor protection of users' authentication information. To ensure the protection of patients' information and data, we propose a scheme that authenticates users based on the information of both the device and the legitimate user. In this paper, we propose a Robust Authentication Model for Healthcare Users (RAMHU) that provides mutual authentication between server and clients. This model utilizes an Elliptic Curve Integrated Encryption Scheme (ECIES) and PHOTON to achieve strong security and a good overall performance. RAMHU relies on multi pseudonyms, physical address, and one-time password mechanisms to authenticate legitimate users. Moreover, extensive informal and formal security analysis with the automated validation of Internet security protocols and applications (AVISPA) tool demonstrates that our model offers a high level of security in repelling a wide variety of possible attacks.