Tao Wang

Tao Wang, Sylvia Herbert, Sicun Gao

Policy gradient lies at the core of deep reinforcement learning (RL) in continuous domains. Despite much success, it is often observed in practice that RL training with policy gradient can fail for many reasons, even on standard control problems with known solutions. We propose a framework for understanding one inherent limitation of the policy gradient approach: the optimization landscape in the policy space can be extremely non-smooth or fractal for certain classes of MDPs, such that there does not exist gradient to be estimated in the first place. We draw on techniques from chaos theory and non-smooth analysis, and analyze the maximal Lyapunov exponents and Hölder exponents of the policy optimization objectives. Moreover, we develop a practical method that can estimate the local smoothness of objective function from samples to identify when the training process has encountered fractal landscapes. We show experiments to illustrate how some failure cases of policy optimization can be explained by such fractal landscapes.

Pengcheng Hou, Tao Wang, Daniel Cerkoney et al.

Quantum field theory (QFT) for interacting many-electron systems is fundamental to condensed matter physics, yet achieving accurate solutions confronts computational challenges in managing the combinatorial complexity of Feynman diagrams, implementing systematic renormalization, and evaluating high-dimensional integrals. We present a unifying framework that integrates QFT computational workflows with an AI-powered technology stack. A cornerstone of this framework is representing Feynman diagrams as computational graphs, which structures the inherent mathematical complexity and facilitates the application of optimized algorithms developed for machine learning and high-performance computing. Consequently, automatic differentiation, native to these graph representations, delivers efficient, fully automated, high-order field-theoretic renormalization procedures. This graph-centric approach also enables sophisticated numerical integration; our neural-network-enhanced Monte Carlo method, accelerated via massively parallel GPU implementation, efficiently evaluates challenging high-dimensional diagrammatic integrals. Applying this framework to the uniform electron gas, we determine the quasiparticle effective mass to a precision significantly surpassing current state-of-the-art simulations. Our work demonstrates the transformative potential of integrating AI-driven computational advances with QFT, opening systematic pathways for solving complex quantum many-body problems across disciplines.