Ruihan Guo

Ruihan Guo, Francis Filbet

The level set method is often used to capture interface behavior in two or three dimensions. In this paper, we present a combination of local discontinuous Galerkin (LDG) method and level set method for simulating Willmore flow. The LDG scheme is energy stable and mass conservative, which are good properties comparing with other numerical methods. In addition, to enhance the efficiency of the proposed LDG scheme and level set method, we employ a p-adaptive local discontinuous Galerkin technique, which applies high order polynomial approximations around the zero level set and low order ones away from the zero level set. A major advantage of the level set method is that the topological changes are well defined and easily performed. In particular, given the stiffness of Willmore flow, a high order semi-implicit Runge-Kutta method is employed for time discretization, which allows larger time step. The equations at the implicit time level are linear, we demonstrate an efficient and practical multi-grid solver to solve the equations. Numerical examples are given to illustrate the combination of the LDG scheme and level set method provides an efficient and practical approach when simulating the Willmore flow.

Ziwen Wang, Jiajun Fan, Ruihan Guo et al.

Protein generative models have shown remarkable promise in protein design but still face limitations in success rate, due to the scarcity of high-quality protein datasets for supervised pretraining. We present ProteinZero, a novel framework that enables scalable, automated, and continuous self-improvement of the inverse folding model through online reinforcement learning. To achieve computationally tractable online feedback, we introduce efficient proxy reward models based on ESM-fold and a novel rapid ddG predictor that significantly accelerates evaluation speed. ProteinZero employs a general RL framework balancing multi-reward maximization, KL-divergence from a reference model, and a novel protein-embedding level diversity regularization that prevents mode collapse while promoting higher sequence diversity. Through extensive experiments, we demonstrate that ProteinZero substantially outperforms existing methods across every key metric in protein design, achieving significant improvements in structural accuracy, designability, thermodynamic stability, and sequence diversity. Most impressively, ProteinZero reduces design failure rates by approximately 36% - 48% compared to widely-used methods like ProteinMPNN, ESM-IF and InstructPLM, consistently achieving success rates exceeding 90% across diverse and complex protein folds. Notably, the entire RL run on CATH-4.3 can be done with a single 8 X GPU node in under 3 days, including reward computation. Our work establishes a new paradigm for protein design where models evolve continuously from their own generated outputs, opening new possibilities for exploring the vast protein design space.

Zhizhou Ren, Ruihan Guo, Yuan Zhou et al.

Many practical applications of reinforcement learning require agents to learn from sparse and delayed rewards. It challenges the ability of agents to attribute their actions to future outcomes. In this paper, we consider the problem formulation of episodic reinforcement learning with trajectory feedback. It refers to an extreme delay of reward signals, in which the agent can only obtain one reward signal at the end of each trajectory. A popular paradigm for this problem setting is learning with a designed auxiliary dense reward function, namely proxy reward, instead of sparse environmental signals. Based on this framework, this paper proposes a novel reward redistribution algorithm, randomized return decomposition (RRD), to learn a proxy reward function for episodic reinforcement learning. We establish a surrogate problem by Monte-Carlo sampling that scales up least-squares-based reward redistribution to long-horizon problems. We analyze our surrogate loss function by connection with existing methods in the literature, which illustrates the algorithmic properties of our approach. In experiments, we extensively evaluate our proposed method on a variety of benchmark tasks with episodic rewards and demonstrate substantial improvement over baseline algorithms.