Shiqin Liu

Haifeng Wang, Hua Wu, Tian Wu et al.

In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, based on an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. To address practical challenges in large-scale deployment under diverse resource constraints, ERNIE 5.0 adopts a novel elastic training paradigm. Within a single pre-training run, the model learns a family of sub-models with varying depths, expert capacities, and routing sparsity, enabling flexible trade-offs among performance, model size, and inference latency in memory- or time-constrained scenarios. Moreover, we systematically address the challenges of scaling reinforcement learning to unified foundation models, thereby guaranteeing efficient and stable post-training under ultra-sparse MoE architectures and diverse multimodal settings. Extensive experiments demonstrate that ERNIE 5.0 achieves strong and balanced performance across multiple modalities. To the best of our knowledge, among publicly disclosed models, ERNIE 5.0 represents the first production-scale realization of a trillion-parameter unified autoregressive model that supports both multimodal understanding and generation. To facilitate further research, we present detailed visualizations of modality-agnostic expert routing in the unified model, alongside comprehensive empirical analysis of elastic training, aiming to offer profound insights to the community.

Shiqin Liu, Haijun Yu

We propose two efficient energetic spectral-element methods in time for marching nonlinear gradient systems with the phase-field Allen--Cahn equation as an example: one fully implicit nonlinear method and one semi-implicit linear method. Different from other spectral methods in time using spectral Petrov-Galerkin or weighted Galerkin approximations, the presented implicit method employs an energetic variational Galerkin form that can maintain the mass conservation and energy dissipation property of the continuous dynamical system. Another advantage of this method is its superconvergence. A high-order extrapolation is adopted for the nonlinear term to get the semi-implicit method. The semi-implicit method does not have superconvergence, but can be improved by a few Picard-like iterations to recover the superconvergence of the implicit method. Numerical experiments verify that the method using Legendre elements of degree three outperforms the 4th-order implicit-explicit backward differentiation formula and the 4th-order exponential time difference Runge-Kutta method, which were known to have best performances in solving phase-field equations. In addition to the standard Allen--Cahn equation, we also apply the method to a conservative Allen--Cahn equation, in which the conservation of discrete total mass is verified. The applications of the proposed methods are not limited to phase-field Allen--Cahn equations. They are suitable for solving general, large-scale nonlinear dynamical systems.