Mingjie Liao

Mingjie Liao, Ping Lin, Lei Zhang

In this paper, we develop the residual based a posteriori error estimates and the corresponding adaptive mesh refinement algorithm for atomistic/continuum (a/c) coupling with finite range interactions in two dimensions. We have systematically derived a new explicitly computable stress tensor formula for finite range interactions. In particular, we use the geometric reconstruction based consistent atomistic/continuum (GRAC) coupling scheme, which is optimal if the continuum model is discretized by $P^1$ finite elements. The numerical results of the adaptive mesh refinement algorithm is consistent with the optimal a priori error estimates.

Hao Wang, Mingjie Liao, Ping Lin et al.

Atomistic/continuum coupling methods aim to achieve optimal balance between accuracy and efficiency. Adaptivity is the key for the efficient implementation of such methods. In this paper, we carry out a rigorous a posteriori analysis of the residual, the stability constant, and the error bound, for a consistent atomistic/continuum coupling method in 2D. We design and implement the corresponding adaptive mesh refinement algorithm, and the convergence rate with respect to degrees of freedom is optimal compare with a priori error estimates.

Mingjie Liao, Ping Lin

A dynamical atomistic chain to simulate mechanical properties of a one-dimensional material with zero temperature may be modelled by the molecular dynamics (MD) model. Because the number of particles (atoms) is huge for a MD model, in practice one often takes a much smaller number of particles to formulate a coarse-grained approximation. We shall mainly consider the consistency of the coarse-grained model with respect to the grain (mesh) size to provide a justification to the goodness of such an approximation. In order to reduce the characteristic oscillations with very different frequencies in such a model, we either add a viscous term to the coarse-grained MD model or apply a space average to the coarse-grained MD solutions for the consistency study. The coarse-grained solution is also compared with the solution of the (macroscopic) continuum model (a nonlinear wave equation of mixed type) to show how well the coarse-grained model can approximate the macroscopic behavior of the material. We also briefly study the instability of the dynamical atomistic chain and the solution of the Riemann problem of the continuum model which may be related to the defect of the atomistic chain under a large deformation in certain locations.

Chenhao Dang, Jing Ma, Mingjie Liao

The composition of training data, governed by the diversity of sources and their mixing strategy, is a cornerstone of Large Language Model (LLM) pre-training. Online Data Mixing (ODM), the technique of adaptively adjusting data mixtures during training, has emerged as a promising direction to improve efficiency. However, existing methods are constrained by their reliance on a singular optimization perspective, which fundamentally overlooks the need for complex LLM pre-training to consider the dynamic data composition from multiple dimensions. To overcome this limitation, we introduce the Holistic Data Scheduler (HDS), a novel online data mixing framework. HDS formulates the data scheduling challenge as a reinforcement learning problem in a continuous control space and leverages the Soft Actor-Critic (SAC) algorithm for its stability and sample efficiency in exploring the high-dimensional policy space. At the core of HDS lies a novel multi-objective, holistic reward function that integrates three critical perspectives: a data-driven reward for quality, a loss-driven reward capturing inter-domain influence, and a model-driven reward based on weight norms. To validate our design and determine its optimal configuration, we conducted systematic experiments on LLMs of various sizes. On The Pile benchmark, HDS reaches the final validation perplexity of the next best method with 44% fewer training iterations. Furthermore, it achieves a 7.2% improvement on the MMLU 0-shot task along with consistent gains on other benchmarks, showcasing its ability to enhance both training efficiency and final model capability.

Jing Ma, Chenhao Dang, Mingjie Liao

The coverage and composition of pretraining data significantly impacts the generalization ability of Large Language Models (LLMs). To reduce the carbon footprint and financial costs of training, some data mixing methods, which applied the optimized domain weights of a small proxy model to train a larger one, were proposed. However, these methods did not evolute with the training dynamics. The existing online data mixing (ODM) method addressed this limitation by applying the multi-armed bandit algorithm as data sampling strategy. Yet, it did not consider the intra-domain interactions. In this paper, we develop an actor-critic based online data mixing (AC-ODM) method, which captures the varying domain weights by auxiliary actor-critic networks and consider the intra-domain interactions with the reward function. While constructing the dataset to pretrain a large target LLM, we directly apply the actor, which is trained with a small proxy LLM as the environment, as the sampling strategy. The transfer of sampling strategy can not only ensure the efficiency of dynamical data mixing, but also expedite the convergence of pretraining the target LLM. Numerical results demonstrate that AC-ODM-410M, which invokes the sampling strategy obtained by a proxy LLM with 410M parameters, reaching the optimal validation perplexity of ODM 71% faster, and improves performance on the zero-shot MMLU benchmark by 27.5% of accuracy, about 2.23x better on pass@1 of HumanEval benchmark.