Jianfeng Lu

Taehoon Kim, Pyunghwan Ahn, Sangyun Kim et al. · nvidia, utoronto

In this report, we introduce NICE (New frontiers for zero-shot Image Captioning Evaluation) project and share the results and outcomes of 2023 challenge. This project is designed to challenge the computer vision community to develop robust image captioning models that advance the state-of-the-art both in terms of accuracy and fairness. Through the challenge, the image captioning models were tested using a new evaluation dataset that includes a large variety of visual concepts from many domains. There was no specific training data provided for the challenge, and therefore the challenge entries were required to adapt to new types of image descriptions that had not been seen during training. This report includes information on the newly proposed NICE dataset, evaluation methods, challenge results, and technical details of top-ranking entries. We expect that the outcomes of the challenge will contribute to the improvement of AI models on various vision-language tasks.

Yuejia Zhang, Zhe Wang, Jianfeng Lu et al.

CDFCI is a shared-memory parallel numerical program for computing low-lying eigenpairs of large-scale, non-relativistic fermionic Hamiltonians. The software is designed to handle a broad class of many-body quantum models, including both ab initio electronic structure Hamiltonians and lattice-based Hamiltonians arising in condensed matter physics. CDFCI combines an efficient coordinate-descent-based selected configuration interaction algorithm with dedicated parallelization strategies, achieving high performance on modern multi-core architectures. Benchmark results on representative quantum chemistry and condensed matter test cases demonstrate that CDFCI attains state-of-the-art accuracy with competitive performance compared to established selected configuration interaction (such as CIPSI or SHCI) and DMRG implementations. The software is open-source, extensively documented, and provides a Python interface for seamless integration with PySCF and other many-body simulation workflows.

Yuehaw Khoo, Jianfeng Lu, Lexing Ying

The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modeled into the equations as random coefficients. However, very often the variability of physical quantities derived from a PDE can be captured by a few features on the space of the coefficient fields. Based on such an observation, we propose using a neural-network (NN) based method to parameterize the physical quantity of interest as a function of input coefficients. The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE. We further demonstrate the simplicity and accuracy of the approach through notable examples of PDEs in engineering and physics.

Zhenyuan Zhang, Zhenbo Song, Kaihao Zhang et al.

Deep learning based methods have achieved significant success in the task of single image reflection removal (SIRR). However, the majority of these methods are focused on High-Definition/Standard-Definition (HD/SD) images, while ignoring higher resolution images such as Ultra-High-Definition (UHD) images. With the increasing prevalence of UHD images captured by modern devices, in this paper, we aim to address the problem of UHD SIRR. Specifically, we first synthesize two large-scale UHD datasets, UHDRR4K and UHDRR8K. The UHDRR4K dataset consists of $2,999$ and $168$ quadruplets of images for training and testing respectively, and the UHDRR8K dataset contains $1,014$ and $105$ quadruplets. To the best of our knowledge, these two datasets are the first largest-scale UHD datasets for SIRR. Then, we conduct a comprehensive evaluation of six state-of-the-art SIRR methods using the proposed datasets. Based on the results, we provide detailed discussions regarding the strengths and limitations of these methods when applied to UHD images. Finally, we present a transformer-based architecture named RRFormer for reflection removal. RRFormer comprises three modules, namely the Prepossessing Embedding Module, Self-attention Feature Extraction Module, and Multi-scale Spatial Feature Extraction Module. These modules extract hypercolumn features, global and partial attention features, and multi-scale spatial features, respectively. To ensure effective training, we utilize three terms in our loss function: pixel loss, feature loss, and adversarial loss. We demonstrate through experimental results that RRFormer achieves state-of-the-art performance on both the non-UHD dataset and our proposed UHDRR datasets. The code and datasets are publicly available at https://github.com/Liar-zzy/Benchmarking-Ultra-High-Definition-Single-Image-Reflection-Removal.

Lin Lin, Jianfeng Lu, Lexing Ying et al.

Kohn-Sham density functional theory is one of the most widely used electronic structure theories. In the pseudopotential framework, uniform discretization of the Kohn-Sham Hamiltonian generally results in a large number of basis functions per atom in order to resolve the rapid oscillations of the Kohn-Sham orbitals around the nuclei. Previous attempts to reduce the number of basis functions per atom include the usage of atomic orbitals and similar objects, but the atomic orbitals generally require fine tuning in order to reach high accuracy. We present a novel discretization scheme that adaptively and systematically builds the rapid oscillations of the Kohn-Sham orbitals around the nuclei as well as environmental effects into the basis functions. The resulting basis functions are localized in the real space, and are discontinuous in the global domain. The continuous Kohn-Sham orbitals and the electron density are evaluated from the discontinuous basis functions using the discontinuous Galerkin (DG) framework. Our method is implemented in parallel and the current implementation is able to handle systems with at least thousands of atoms. Numerical examples indicate that our method can reach very high accuracy (less than 1meV) with a very small number ($4\sim 40$) of basis functions per atom.

Ziang Chen, Jialin Liu, Xinshang Wang et al.

Learning to optimize is a rapidly growing area that aims to solve optimization problems or improve existing optimization algorithms using machine learning (ML). In particular, the graph neural network (GNN) is considered a suitable ML model for optimization problems whose variables and constraints are permutation--invariant, for example, the linear program (LP). While the literature has reported encouraging numerical results, this paper establishes the theoretical foundation of applying GNNs to solving LPs. Given any size limit of LPs, we construct a GNN that maps different LPs to different outputs. We show that properly built GNNs can reliably predict feasibility, boundedness, and an optimal solution for each LP in a broad class. Our proofs are based upon the recently--discovered connections between the Weisfeiler--Lehman isomorphism test and the GNN. To validate our results, we train a simple GNN and present its accuracy in mapping LPs to their feasibilities and solutions.

Lin Lin, Jianfeng Lu, Roberto Car et al.

We propose a multipole representation of the Fermi-Dirac function and the Fermi operator, and use this representation to develop algorithms for electronic structure analysis of metallic systems. The new algorithm is quite simple and efficient. Its computational cost scales logarithmically with $βΔ\eps$ where $β$ is the inverse temperature, and $Δ\eps$ is the width of the spectrum of the discretized Hamiltonian matrix.

Qiang Du, Xingjie Helen Li, Jianfeng Lu et al.

In this paper, we extend the idea of "geometric reconstruction" to couple a nonlocal diffusion model directly with the classical local diffusion in one dimensional space. This new coupling framework removes interfacial inconsistency, ensures the flux balance, and satisfies energy conservation as well as the maximum principle, whereas none of existing coupling methods for nonlocal-to-local coupling satisfies all of these properties. We establish the well-posedness and provide the stability analysis of the coupling method. We investigate the difference to the local limiting problem in terms of the nonlocal interaction range. Furthermore, we propose a first order finite difference numerical discretization and perform several numerical tests to confirm the theoretical findings. In particular, we show that the resulting numerical result is free of artifacts near the boundary of the domain where a classical local boundary condition is used, together with a coupled fully nonlocal model in the interior of the domain.

Ziang Chen, Jialin Liu, Xinshang Wang et al.

While Mixed-integer linear programming (MILP) is NP-hard in general, practical MILP has received roughly 100--fold speedup in the past twenty years. Still, many classes of MILPs quickly become unsolvable as their sizes increase, motivating researchers to seek new acceleration techniques for MILPs. With deep learning, they have obtained strong empirical results, and many results were obtained by applying graph neural networks (GNNs) to making decisions in various stages of MILP solution processes. This work discovers a fundamental limitation: there exist feasible and infeasible MILPs that all GNNs will, however, treat equally, indicating GNN's lacking power to express general MILPs. Then, we show that, by restricting the MILPs to unfoldable ones or by adding random features, there exist GNNs that can reliably predict MILP feasibility, optimal objective values, and optimal solutions up to prescribed precision. We conducted small-scale numerical experiments to validate our theoretical findings.

Han Sun, Zhaoxin Fan, Zhenbo Song et al.

Monocular 3D object detection is a fundamental but very important task to many applications including autonomous driving, robotic grasping and augmented reality. Existing leading methods tend to estimate the depth of the input image first, and detect the 3D object based on point cloud. This routine suffers from the inherent gap between depth estimation and object detection. Besides, the prediction error accumulation would also affect the performance. In this paper, a novel method named MonoSIM is proposed. The insight behind introducing MonoSIM is that we propose to simulate the feature learning behaviors of a point cloud based detector for monocular detector during the training period. Hence, during inference period, the learned features and prediction would be similar to the point cloud based detector as possible. To achieve it, we propose one scene-level simulation module, one RoI-level simulation module and one response-level simulation module, which are progressively used for the detector's full feature learning and prediction pipeline. We apply our method to the famous M3D-RPN detector and CaDDN detector, conducting extensive experiments on KITTI and Waymo Open datasets. Results show that our method consistently improves the performance of different monocular detectors for a large margin without changing their network architectures. Our codes will be publicly available at https://github.com/sunh18/MonoSIM}{https://github.com/sunh18/MonoSIM.

Hongrui Chen, Holden Lee, Jianfeng Lu

We give an improved theoretical analysis of score-based generative modeling. Under a score estimate with small $L^2$ error (averaged across timesteps), we provide efficient convergence guarantees for any data distribution with second-order moment, by either employing early stopping or assuming smoothness condition on the score function of the data distribution. Our result does not rely on any log-concavity or functional inequality assumption and has a logarithmic dependence on the smoothness. In particular, we show that under only a finite second moment condition, approximating the following in reverse KL divergence in $ε$-accuracy can be done in $\tilde O\left(\frac{d \log (1/δ)}ε\right)$ steps: 1) the variance-$δ$ Gaussian perturbation of any data distribution; 2) data distributions with $1/δ$-smooth score functions. Our analysis also provides a quantitative comparison between different discrete approximations and may guide the choice of discretization points in practice.

Tanya Marwah, Ashwini Pokle, J. Zico Kolter et al.

Data-driven machine learning approaches are being increasingly used to solve partial differential equations (PDEs). They have shown particularly striking successes when training an operator, which takes as input a PDE in some family, and outputs its solution. However, the architectural design space, especially given structural knowledge of the PDE family of interest, is still poorly understood. We seek to remedy this gap by studying the benefits of weight-tied neural network architectures for steady-state PDEs. To achieve this, we first demonstrate that the solution of most steady-state PDEs can be expressed as a fixed point of a non-linear operator. Motivated by this observation, we propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE as the infinite-depth fixed point of an implicit operator layer using a black-box root solver and differentiates analytically through this fixed point resulting in $\mathcal{O}(1)$ training memory. Our experiments indicate that FNO-DEQ-based architectures outperform FNO-based baselines with $4\times$ the number of parameters in predicting the solution to steady-state PDEs such as Darcy Flow and steady-state incompressible Navier-Stokes. Finally, we show FNO-DEQ is more robust when trained with datasets with more noisy observations than the FNO-based baselines, demonstrating the benefits of using appropriate inductive biases in architectural design for different neural network based PDE solvers. Further, we show a universal approximation result that demonstrates that FNO-DEQ can approximate the solution to any steady-state PDE that can be written as a fixed point equation.

Holden Lee, Jianfeng Lu, Yixin Tan

Score-based generative modeling (SGM) is a highly successful approach for learning a probability distribution from data and generating further samples. We prove the first polynomial convergence guarantees for the core mechanic behind SGM: drawing samples from a probability density $p$ given a score estimate (an estimate of $\nabla \ln p$) that is accurate in $L^2(p)$. Compared to previous works, we do not incur error that grows exponentially in time or that suffers from a curse of dimensionality. Our guarantee works for any smooth distribution and depends polynomially on its log-Sobolev constant. Using our guarantee, we give a theoretical analysis of score-based generative modeling, which transforms white-noise input into samples from a learned data distribution given score estimates at different noise scales. Our analysis gives theoretical grounding to the observation that an annealed procedure is required in practice to generate good samples, as our proof depends essentially on using annealing to obtain a warm start at each step. Moreover, we show that a predictor-corrector algorithm gives better convergence than using either portion alone.

Holden Lee, Jianfeng Lu, Yixin Tan

Score-based generative modeling (SGM) has grown to be a hugely successful method for learning to generate samples from complex data distributions such as that of images and audio. It is based on evolving an SDE that transforms white noise into a sample from the learned distribution, using estimates of the score function, or gradient log-pdf. Previous convergence analyses for these methods have suffered either from strong assumptions on the data distribution or exponential dependencies, and hence fail to give efficient guarantees for the multimodal and non-smooth distributions that arise in practice and for which good empirical performance is observed. We consider a popular kind of SGM -- denoising diffusion models -- and give polynomial convergence guarantees for general data distributions, with no assumptions related to functional inequalities or smoothness. Assuming $L^2$-accurate score estimates, we obtain Wasserstein distance guarantees for any distribution of bounded support or sufficiently decaying tails, as well as TV guarantees for distributions with further smoothness assumptions.

Anton Martinsson, Jianfeng Lu, Benedict Leimkuhler et al.

We investigate the theoretical foundations of the simulated tempering method and use our findings to design efficient algorithms. Employing a large deviation argument first used for replica exchange molecular dynamics [Plattner et al., J. Chem. Phys. 135:134111 (2011)], we demonstrate that the most efficient approach to simulated tempering is to vary the temperature infinitely rapidly. In this limit, we can replace the equations of motion for the temperature and physical variables by averaged equations for the latter alone, with the forces rescaled according to a position-dependent function defined in terms of temperature weights. The averaged equations are similar to those used in Gao's integrated-over-temperature method, except that we show that it is better to use a continuous rather than a discrete set of temperatures. We give a theoretical argument for the choice of the temperature weights as the reciprocal partition function, thereby relating simulated tempering to Wang-Landau sampling. Finally, we describe a self-consistent algorithm for simultaneously sampling the canonical ensemble and learning the weights during simulation. This algorithm is tested on a system of harmonic oscillators as well as a continuous variant of the Curie-Weiss model, where it is shown to perform well and to accurately capture the second-order phase transition observed in this model.

Ke Chen, Qin Li, Jianfeng Lu et al.

In the framework of generalized finite element methods for elliptic equations with rough coefficients, efficiency and accuracy of the numerical method depend critically on the use of appropriate basis functions. This work explores several random sampling strategies that construct approximations to the optimal set of basis functions of a given dimension, and proposes a quantitative criterion to analyze and compare these sampling strategies. Numerical evidence shows that the best results are achieved by two strategies, Random Gaussian and Smooth boundary sampling.

Tanya Marwah, Zachary C. Lipton, Jianfeng Lu et al.

A burgeoning line of research leverages deep neural networks to approximate the solutions to high dimensional PDEs, opening lines of theoretical inquiry focused on explaining how it is that these models appear to evade the curse of dimensionality. However, most prior theoretical analyses have been limited to linear PDEs. In this work, we take a step towards studying the representational power of neural networks for approximating solutions to nonlinear PDEs. We focus on a class of PDEs known as \emph{nonlinear elliptic variational PDEs}, whose solutions minimize an \emph{Euler-Lagrange} energy functional $\mathcal{E}(u) = \int_ΩL(x, u(x), \nabla u(x)) - f(x) u(x)dx$. We show that if composing a function with Barron norm $b$ with partial derivatives of $L$ produces a function of Barron norm at most $B_L b^p$, the solution to the PDE can be $ε$-approximated in the $L^2$ sense by a function with Barron norm $O\left(\left(dB_L\right)^{\max\{p \log(1/ ε), p^{\log(1/ε)}\}}\right)$. By a classical result due to Barron [1993], this correspondingly bounds the size of a 2-layer neural network needed to approximate the solution. Treating $p, ε, B_L$ as constants, this quantity is polynomial in dimension, thus showing neural networks can evade the curse of dimensionality. Our proof technique involves neurally simulating (preconditioned) gradient in an appropriate Hilbert space, which converges exponentially fast to the solution of the PDE, and such that we can bound the increase of the Barron norm at each iterate. Our results subsume and substantially generalize analogous prior results for linear elliptic PDEs over a unit hypercube.

Jianfeng Lu, Eric Vanden-Eijnden

A variant of the parallel tempering method is proposed in terms of a stochastic switching process for the coupled dynamics of replica configuration and temperature permutation. This formulation is shown to facilitate the analysis of the convergence properties of parallel tempering by large deviation theory, which indicates that the method should be operated in the infinite swapping limit to maximize sampling efficiency. The effective equation for the replica alone that arises in this infinite swapping limit simply involves replacing the original potential by a mixture potential. The analysis of the geometric properties of this potential offers a new perspective on the issues of how to choose of temperature ladder, and why many temperatures should typically be introduced to boost the sampling efficiency. It is also shown how to simulate the effective equation in this many temperature regime using multiscale integrators. Finally, similar ideas are also used to discuss extensions of the infinite swapping limits to the technique of simulated tempering.

Lin Lin, Jianfeng Lu, Lexing Ying et al.

We develop a technique for generating a set of optimized local basis functions to solve models in the Kohn-Sham density functional theory for both insulating and metallic systems. The optimized local basis functions are obtained by solving a minimization problem in an admissible set determined by a large number of primitive basis functions. Using the optimized local basis set, the electron energy and the atomic force can be calculated accurately with a small number of basis functions. The Pulay force is systematically controlled and is not required to be calculated, which makes the optimized local basis set an ideal tool for ab initio molecular dynamics and structure optimization. We also propose a preconditioned Newton-GMRES method to obtain the optimized local basis functions in practice. The optimized local basis set is able to achieve high accuracy with a small number of basis functions per atom when applied to a one dimensional model problem.

Dong Wei, Huaijiang Sun, Bin Li et al.

Stochastic human motion prediction aims to forecast multiple plausible future motions given a single pose sequence from the past. Most previous works focus on designing elaborate losses to improve the accuracy, while the diversity is typically characterized by randomly sampling a set of latent variables from the latent prior, which is then decoded into possible motions. This joint training of sampling and decoding, however, suffers from posterior collapse as the learned latent variables tend to be ignored by a strong decoder, leading to limited diversity. Alternatively, inspired by the diffusion process in nonequilibrium thermodynamics, we propose MotionDiff, a diffusion probabilistic model to treat the kinematics of human joints as heated particles, which will diffuse from original states to a noise distribution. This process offers a natural way to obtain the "whitened" latents without any trainable parameters, and human motion prediction can be regarded as the reverse diffusion process that converts the noise distribution into realistic future motions conditioned on the observed sequence. Specifically, MotionDiff consists of two parts: a spatial-temporal transformer-based diffusion network to generate diverse yet plausible motions, and a graph convolutional network to further refine the outputs. Experimental results on two datasets demonstrate that our model yields the competitive performance in terms of both accuracy and diversity.

Jianfeng Lu, Lexing Ying

We propose an efficient algorithm for density fitting of Bloch waves for Hamiltonian operators with periodic potential. The algorithm is based on column selection and random Fourier projection of the orbital functions. The computational cost of the algorithm scales as $\mathcal{O}\bigl(N_{\text{grid}} N^2 + N_{\text{grid}} NK \log (NK)\bigr)$, where $N_{\text{grid}}$ is number of spatial grid points, $K$ is the number of sampling $k$-points in first Brillouin zone, and $N$ is the number of bands under consideration. We validate the algorithm by numerical examples in both two and three dimensions.

Qin Li, Jianfeng Lu, Weiran Sun

We give a unified proof for the well-posedness of a class of linear half-space equations with general incoming data and construct a Galerkin method to numerically resolve this type of equations in a systematic way. Our main strategy in both analysis and numerics includes three steps: adding damping terms to the original half-space equation, using an inf-sup argument and even-odd decomposition to establish the well-posedness of the damped equation, and then recovering solutions to the original half-space equation. The proposed numerical methods for the damped equation is shown to be quasi-optimal and the numerical error of approximations to the original equation is controlled by that of the damped equation. This efficient solution to the half-space problem is useful for kinetic-fluid coupling simulations.

Shijun Zhang, Jianfeng Lu, Hongkai Zhao

This paper explores the expressive power of deep neural networks for a diverse range of activation functions. An activation function set $\mathscr{A}$ is defined to encompass the majority of commonly used activation functions, such as $\mathtt{ReLU}$, $\mathtt{LeakyReLU}$, $\mathtt{ReLU}^2$, $\mathtt{ELU}$, $\mathtt{CELU}$, $\mathtt{SELU}$, $\mathtt{Softplus}$, $\mathtt{GELU}$, $\mathtt{SiLU}$, $\mathtt{Swish}$, $\mathtt{Mish}$, $\mathtt{Sigmoid}$, $\mathtt{Tanh}$, $\mathtt{Arctan}$, $\mathtt{Softsign}$, $\mathtt{dSiLU}$, and $\mathtt{SRS}$. We demonstrate that for any activation function $\varrho\in \mathscr{A}$, a $\mathtt{ReLU}$ network of width $N$ and depth $L$ can be approximated to arbitrary precision by a $\varrho$-activated network of width $3N$ and depth $2L$ on any bounded set. This finding enables the extension of most approximation results achieved with $\mathtt{ReLU}$ networks to a wide variety of other activation functions, albeit with slightly increased constants. Significantly, we establish that the (width,$\,$depth) scaling factors can be further reduced from $(3,2)$ to $(1,1)$ if $\varrho$ falls within a specific subset of $\mathscr{A}$. This subset includes activation functions such as $\mathtt{ELU}$, $\mathtt{CELU}$, $\mathtt{SELU}$, $\mathtt{Softplus}$, $\mathtt{GELU}$, $\mathtt{SiLU}$, $\mathtt{Swish}$, and $\mathtt{Mish}$.

Jianfeng Lu, Felix Otto

We are given a uniformly elliptic coefficient field that we regard as a realization of a stationary and finite-range (say, range unity) ensemble of coefficient fields. Given a (deterministic) right-hand-side supported in a ball of size $\ell\gg 1$ and of vanishing average, we are interested in an algorithm to compute the (gradient of the) solution near the origin, just using the knowledge of the (given realization of the) coefficient field in some large box of size $L\gg\ell$. More precisely, we are interested in the most seamless (artificial) boundary condition on the boundary of the computational domain of size $L$. Motivated by the recently introduced multipole expansion in random media, we propose an algorithm. We rigorously establish an error estimate (on the level of the gradient) in terms of $L\gg\ell\gg 1$, using recent results in quantitative stochastic homogenization. More precisely, our error estimate has an a priori and an a posteriori aspect: With a priori overwhelming probability, the (random) prefactor can be bounded by a constant that is computable without much further effort, on the basis of the given realization in the box of size $L$. We also rigorously establish that the order of the error estimate in both $L$ and $\ell$ is optimal, where in this paper we focus on the case of $d=2$. This amounts to a lower bound on the variance of the quantity of interest when conditioned on the coefficients inside the computational domain, and relies on the deterministic insight that a sensitivity analysis wrt a defect commutes with (stochastic) homogenization. Finally, we carry out numerical experiments that show that this optimal convergence rate already sets in at only moderately large $L$, and that more naive boundary conditions perform worse both in terms of rate and prefactor.

Jianfeng Lu, Zhennan Zhou

In this work, a novel ring polymer representation for multi-level quantum system is proposed for thermal average calculations. The proposed presentation keeps the discreteness of the electronic states: besides position and momentum, each bead in the ring polymer is also characterized by a surface index indicating the electronic energy surface. A path integral molecular dynamics with surface hopping (PIMD-SH) dynamics is also developed to sample the equilibrium distribution of ring polymer configurational space. The PIMD-SH sampling method is validated theoretically and by numerical examples.

Qin Li, Jianfeng Lu, Weiran Sun

Recent result [Wu and Guo, Comm. Math. Phys., 2015] has shown that over the 2D unit disk, the classical half-space equation (CHS) for the neutron transport does not capture the correct boundary layer behaviour as long believed. In this paper we develop a regularization technique for CHS to any arbitrary order and use its first-order regularization to show that in the case of the 2D unit disk, although CHS misrepresents the boundary layer behaviour, it does give the correct boundary condition for the interior macroscopic (Laplace) equation. Therefore CHS is still a valid equation to recover the correct boundary condition for the interior Laplace equation over the 2D unit disk.

Jianfeng Lu, Pingbing Ming

We study a force-based hybrid method that couples atomistic models with nonlinear Cauchy-Born elasticity models. We show that the proposed scheme converges quadratically to the solution of the atomistic model, as the ratio between lattice parameter and the characteristic length scale of the deformation tends to zero. Convergence is established for general short-ranged atomistic potential and for simple lattices in three dimension. The convergence is based on consistency and stability analysis. General tools are developed in the framework of pseudo-difference operators for stability analysis in arbitrary dimension of the multiscale atomistic and continuum coupling methods.

Jianfeng Lu, Stefan Steinerberger

We describe a way of detecting the location of localized eigenvectors of a linear system $Ax = λx$ for eigenvalues $λ$ with $|λ|$ comparatively large. We define the family of functions $f_α: \left\{1.2. \dots, n\right\} \rightarrow \mathbb{R}_{}$ $$ f_α(k) = \log \left( \| A^α e_k \|_{\ell^2} \right),$$ where $α\geq 0$ is a parameter and $e_k = (0,0,\dots, 0,1,0, \dots, 0)$ is the $k-$th standard basis vector. We prove that eigenvectors associated to eigenvalues with large absolute value localize around local maxima of $f_α$: the metastable states in the power iteration method (slowing down its convergence) can be used to predict localization. We present a fast randomized algorithm and discuss different examples: a random band matrix, discretizations of the local operator $-Δ+ V$ and the nonlocal operator $(-Δ)^{3/4} + V$.

Jianfeng Lu, Lexing Ying

We develop a robust and efficient method for soliton calculations for nonlinear Schrödinger equations. The method is based on the recently developed sparsifying preconditioner combined with Newton's iterative method. The performance of the method is demonstrated by numerical examples of gap solitons in the context of nonlinear optics.

Jianfeng Lu, Zhennan Zhou

We develop a surface hopping algorithm based on frozen Gaussian approximation for semiclassical matrix Schrödinger equations, in the spirit of Tully's fewest switches surface hopping method. The algorithm is asymptotically derived from the Schrödinger equation with rigorous approximation error analysis. The resulting algorithm can be viewed as a path integral stochastic representation of the semiclassical matrix Schrödinger equations. Our results provide mathematical understanding to and shed new light on the important class of surface hopping methods in theoretical and computational chemistry.

Jianfeng Lu, Yue Wu, Yang Xiang

We use the score-based transport modeling method to solve the mean-field Fokker-Planck equations, which we call MSBTM. We establish an upper bound on the time derivative of the Kullback-Leibler (KL) divergence to MSBTM numerical estimation from the exact solution, thus validates the MSBTM approach. Besides, we provide an error analysis for the algorithm. In numerical experiments, we study two types of mean-field Fokker-Planck equation and their corresponding dynamics of particles in interacting systems. The MSBTM algorithm is numerically validated through qualitative and quantitative comparison between the MSBTM solutions, the results of integrating the associated stochastic differential equation and the analytical solutions if available.

Ye He, Krishnakumar Balasubramanian, Bharath K. Sriperumbudur et al.

The Stein Variational Gradient Descent (SVGD) algorithm is a deterministic particle method for sampling. However, a mean-field analysis reveals that the gradient flow corresponding to the SVGD algorithm (i.e., the Stein Variational Gradient Flow) only provides a constant-order approximation to the Wasserstein Gradient Flow corresponding to the KL-divergence minimization. In this work, we propose the Regularized Stein Variational Gradient Flow, which interpolates between the Stein Variational Gradient Flow and the Wasserstein Gradient Flow. We establish various theoretical properties of the Regularized Stein Variational Gradient Flow (and its time-discretization) including convergence to equilibrium, existence and uniqueness of weak solutions, and stability of the solutions. We provide preliminary numerical evidence of the improved performance offered by the regularization.

Rongjie Lai, Jianfeng Lu

The committor functions provide useful information to the understanding of transitions of a stochastic system between disjoint regions in phase space. In this work, we develop a point cloud discretization for Fokker-Planck operators to numerically calculate the committor function, with the assumption that the transition occurs on an intrinsically low-dimensional manifold in the ambient potentially high dimensional configurational space of the stochastic system. Numerical examples on model systems validate the effectiveness of the proposed method.

Jianfeng Lu, Zhe Wang

In this paper, we propose a general analysis framework for inexact power iteration, which can be used to efficiently solve high dimensional eigenvalue problems arising from quantum many-body problems. Under the proposed framework, we establish the convergence theorems for several recently proposed randomized algorithms, including the full configuration interaction quantum Monte Carlo (FCIQMC) and the fast randomized iteration (FRI). The analysis is consistent with numerical experiments for physical systems such as Hubbard model and small chemical molecules. We also compare the algorithms both in convergence analysis and numerical results.

Jianfeng Lu, Kyle Thicke

We consider a modification of the OMM energy functional which contains an $\ell^1$ penalty term in order to find a sparse representation of the low-lying eigenspace of self-adjoint operators. We analyze the local minima of the modified functional as well as the convergence of the modified functional to the original functional. Algorithms combining soft thresholding with gradient descent are proposed for minimizing this new functional. Numerical tests validate our approach. As an added bonus, we also prove the unanticipated and remarkable property that every local minimum the OMM functional without the $\ell^1$ term is also a global minimum.

Zhenbo Song, Xianghui Ze, Jianfeng Lu et al.

This paper addresses the problem of estimating the 3-DoF camera pose for a ground-level image with respect to a satellite image that encompasses the local surroundings. We propose a novel end-to-end approach that leverages the learning of dense pixel-wise flow fields in pairs of ground and satellite images to calculate the camera pose. Our approach differs from existing methods by constructing the feature metric at the pixel level, enabling full-image supervision for learning distinctive geometric configurations and visual appearances across views. Specifically, our method employs two distinct convolution networks for ground and satellite feature extraction. Then, we project the ground feature map to the bird's eye view (BEV) using a fixed camera height assumption to achieve preliminary geometric alignment. To further establish content association between the BEV and satellite features, we introduce a residual convolution block to refine the projected BEV feature. Optical flow estimation is performed on the refined BEV feature map and the satellite feature map using flow decoder networks based on RAFT. After obtaining dense flow correspondences, we apply the least square method to filter matching inliers and regress the ground camera pose. Extensive experiments demonstrate significant improvements compared to state-of-the-art methods. Notably, our approach reduces the median localization error by 89%, 19%, 80% and 35% on the KITTI, Ford multi-AV, VIGOR and Oxford RobotCar datasets, respectively.

Yu Cao, Jianfeng Lu

In this paper, we extend the dynamical low-rank approximation method to the space of finite signed measures. Under this framework, we derive stochastic low-rank dynamics for stochastic differential equations (SDEs) coming from classical stochastic dynamics or unraveling of Lindblad quantum master equations. We justify the proposed method by error analysis and also numerical examples for applications in solving high-dimensional SDE, stochastic Burgers' equation, and high-dimensional Lindblad equation.

Jianfeng Lu, Haizhao Yang

We present an efficient preconditioner for the orbital minimization method when the Hamiltonian is discretized using planewaves (i.e., pseudospectral method). This novel preconditioner is based on an approximate Fermi operator projection by pole expansion, combined with the sparsifying preconditioner to efficiently evaluate the pole expansion for a wide range of Hamiltonian operators. Numerical results validate the performance of the new preconditioner for the orbital minimization method, in particular, the iteration number is reduced to $O(1)$ and often only a few iterations are enough for convergence.

Jianfeng Lu, Pingbing Ming

We study a force-based hybrid method that couples atomistic model with Cauchy-Born elasticity model with sharp transition interface. We identify stability conditions that guarantee the convergence of the hybrid scheme to the solution of the atomistic model with second order accuracy, as the ratio between lattice parameter and the characteristic length scale of the deformation tends to zero. Convergence is established for hybrid schemes with planar sharp interface for system without defects, with general finite range atomistic potential and simple lattice structure. The key ingredient of the proof is regularity and stability analysis of elliptic systems of difference equations. We apply the results to atomistic-to-continuum scheme for a 2D triangular lattice with planar interface.

Guanyiman Fu, Jingtao Li, Zihang Cheng et al.

While hyperspectral imaging provides rich spatial-spectral information across hundreds of narrow wavelength bands for precise material identification, ground-based hyperspectral pre-trained backbones remain absent, constrained by varying spectral configurations across sensors, the scarcity and inconsistency of labels, and the limited scale and scene diversity of existing datasets. To address these challenges and enable universal perception, we propose HyperVision, the first ground-based hyperspectral pre-trained backbone. First, to handle varying spectral configurations, HyperVision adopts a channel-adaptive dynamic embedding mechanism to map heterogeneous inputs into a unified token space. Second, to address the scarcity and inconsistency of labels, we introduce a multi-source pseudo-labeling method that fuses semantic representations from both spatial structures generated by SAM2 and fine-grained spectral material information extracted by HyperFree. Third, to compensate for limited dataset scale and enrich scene diversity, a cross-modal knowledge distillation mechanism is utilized to transfer rich semantic representations from a pre-trained RGB vision model to our hyperspectral backbone. Pre-trained on a collection of 15k images from 26 diverse ground-based datasets, HyperVision demonstrates exceptional generalization. Requiring only efficient head-only adaptation without adjusting backbone parameters, it achieves state-of-the-art performance compared to task-specific methods across three downstream tasks under varying sensor configurations, yielding up to a 16.3% relative improvement in hyperspectral semantic segmentation $\mathrm{Acc}_{\mathrm{M}}$, a 2.1% relative gain in object tracking AUC, and a 35.5% reduction in salient object detection MAE. The source code and pre-trained model will be publicly available at https://github.com/lronkitty/HyperVision .

Jianfeng Lu, Zhennan Zhou

To accelerate the thermal equilibrium sampling of multi-level quantum systems, the infinite swapping limit of a recently proposed multi-level ring polymer representation is investigated. In the infinite swapping limit, the ring polymer evolves according to an averaged Hamiltonian with respect to all possible surface index configurations of the ring polymer, thus connects the surface hopping approach to the mean-field path integral molecular dynamics. A multiscale integrator for the infinite swapping limit is also proposed to enable efficient sampling based on the limiting dynamics. Numerical results demonstrate the huge improvement of sampling efficiency of the infinite swapping compared with the direct simulation of path integral molecular dynamics with surface hopping.

Xiantao Li, Lin Lin, Jianfeng Lu

In this paper, we propose a new Green's function embedding method called PEXSI-$Σ$ for describing complex systems within the Kohn-Sham density functional theory (KSDFT) framework, after revisiting the physics literature of Green's function embedding methods from a numerical linear algebra perspective. The PEXSI-$Σ$ method approximates the density matrix using a set of nearly optimally chosen Green's functions evaluated at complex frequencies. For each Green's function, the complex boundary conditions are described by a self energy matrix $Σ$ constructed from a physical reference Green's function, which can be computed relatively easily. In the linear regime, such treatment of the boundary condition can be numerically exact. The support of the $Σ$ matrix is restricted to degrees of freedom near the boundary of computational domain, and can be interpreted as a frequency dependent surface potential. This makes it possible to perform KSDFT calculations with $\mathcal{O}(N^2)$ computational complexity, where $N$ is the number of atoms within the computational domain. Green's function embedding methods are also naturally compatible with atomistic Green's function methods for relaxing the atomic configuration outside the computational domain. As a proof of concept, we demonstrate the accuracy of the PEXSI-$Σ$ method for graphene with divacancy and dislocation dipole type of defects using the DFTB+ software package.

Zhiyi You, Liying Li, Jianfeng Lu et al.

Fast and accurate sampling method is in high demand, in order to bridge the large gaps between molecular dynamic simulations and experimental observations. Recently, integrated tempering enhanced sampling method (ITS) has been proposed and successfully applied to various biophysical examples, significantly accelerating conformational sampling. The mathematical validation for its effectiveness has not been elucidated yet. Here we show that the integrated tempering enhanced sampling method can be viewed as a reformulation of the infinite switching limit of simulated tempering method over a mixed potential. Moreover, we demonstrate that the efficiency of simulated tempering molecular dynamics (STMD) improves as the frequency of switching between the temperatures is increased, based on the large deviation principle of empirical distributions. Our theory provides the theoretical justification of the advantage of ITS. Finally, we illustrate the utility of the infinite switching simulated tempering method through several numerical examples.

Yingzhou Li, Jianfeng Lu

This work aims at understanding of bold diagrammatic Monte Carlo (BDMC) methods for stochastic summation of Feynman diagrams from the angle of stochastic iterative methods. The convergence enhancement trick of the BDMC is investigated from the analysis of condition number and convergence of the stochastic iterative methods. Numerical experiments are carried out for model systems to compare the BDMC with related stochastic iterative approaches.

Hongxu Chen, Qin Li, Jianfeng Lu

The Bhatnagar-Gross-Krook (BGK) model, a simplification of the Boltzmann equation, in the absence of boundary effect, converges to the Euler equations when the Knudsen number is small. In practice, however, Knudsen layers emerge at the physical boundary, or at the interfaces between the two regimes. We model the Knudsen layer using a half-space kinetic equation, and apply a half-space numerical solver [ESAIM: M2AN 51 (2017) 1583-1615] [Math. Comp. 86 (2017), 1269-1301] to quantify the transition between the kinetic to the fluid regime. A full domain numerical solver is developed with a domain-decomposition approach, where we apply the Euler solver and kinetic solver on the appropriate subdomains and connect them via the half-space solver. In the nonlinear case, linearization is performed upon local Maxwellian. Despite the lack of analytical support, the numerical evidence nevertheless demonstrates that the linearization approach is promising.

Xiaoning Sun, Huaijiang Sun, Bin Li et al.

Let us rethink the real-world scenarios that require human motion prediction techniques, such as human-robot collaboration. Current works simplify the task of predicting human motions into a one-off process of forecasting a short future sequence (usually no longer than 1 second) based on a historical observed one. However, such simplification may fail to meet practical needs due to the neglect of the fact that motion prediction in real applications is not an isolated ``observe then predict'' unit, but a consecutive process composed of many rounds of such unit, semi-overlapped along the entire sequence. As time goes on, the predicted part of previous round has its corresponding ground truth observable in the new round, but their deviation in-between is neither exploited nor able to be captured by existing isolated learning fashion. In this paper, we propose DeFeeNet, a simple yet effective network that can be added on existing one-off prediction models to realize deviation perception and feedback when applied to consecutive motion prediction task. At each prediction round, the deviation generated by previous unit is first encoded by our DeFeeNet, and then incorporated into the existing predictor to enable a deviation-aware prediction manner, which, for the first time, allows for information transmit across adjacent prediction units. We design two versions of DeFeeNet as MLP-based and GRU-based, respectively. On Human3.6M and more complicated BABEL, experimental results indicate that our proposed network improves consecutive human motion prediction performance regardless of the basic model.

Shijun Zhang, Jianfeng Lu, Hongkai Zhao

This paper explores the expressive power of deep neural networks through the framework of function compositions. We demonstrate that the repeated compositions of a single fixed-size ReLU network exhibit surprising expressive power, despite the limited expressive capabilities of the individual network itself. Specifically, we prove by construction that $\mathcal{L}_2\circ \boldsymbol{g}^{\circ r}\circ \boldsymbol{\mathcal{L}}_1$ can approximate $1$-Lipschitz continuous functions on $[0,1]^d$ with an error $\mathcal{O}(r^{-1/d})$, where $\boldsymbol{g}$ is realized by a fixed-size ReLU network, $\boldsymbol{\mathcal{L}}_1$ and $\mathcal{L}_2$ are two affine linear maps matching the dimensions, and $\boldsymbol{g}^{\circ r}$ denotes the $r$-times composition of $\boldsymbol{g}$. Furthermore, we extend such a result to generic continuous functions on $[0,1]^d$ with the approximation error characterized by the modulus of continuity. Our results reveal that a continuous-depth network generated via a dynamical system has immense approximation power even if its dynamics function is time-independent and realized by a fixed-size ReLU network.

Qin Li, Jianfeng Lu

We design a numerical scheme for transport equations with oscillatory periodic scattering coefficients. The scheme is asymptotic preserving in the diffusion limit as Knudsen number goes to zero. It also captures the homogenization limit as the length scale of the scattering coefficient goes to zero. The proposed method is based on the construction of multiscale finite element basis and a Galerkin projection based on the even-odd decomposition. The method is analyzed in the asymptotic regime, as well as validated numerically.

Qiongjie Cui, Huaijiang Sun, Jianfeng Lu et al.

Predicting high-fidelity future human poses, from a historically observed sequence, is decisive for intelligent robots to interact with humans. Deep end-to-end learning approaches, which typically train a generic pre-trained model on external datasets and then directly apply it to all test samples, emerge as the dominant solution to solve this issue. Despite encouraging progress, they remain non-optimal, as the unique properties (e.g., motion style, rhythm) of a specific sequence cannot be adapted. More generally, at test-time, once encountering unseen motion categories (out-of-distribution), the predicted poses tend to be unreliable. Motivated by this observation, we propose a novel test-time adaptation framework that leverages two self-supervised auxiliary tasks to help the primary forecasting network adapt to the test sequence. In the testing phase, our model can adjust the model parameters by several gradient updates to improve the generation quality. However, due to catastrophic forgetting, both auxiliary tasks typically tend to the low ability to automatically present the desired positive incentives for the final prediction performance. For this reason, we also propose a meta-auxiliary learning scheme for better adaptation. In terms of general setup, our approach obtains higher accuracy, and under two new experimental designs for out-of-distribution data (unseen subjects and categories), achieves significant improvements.

Andrea Agazzi, Jianfeng Lu, Sayan Mukherjee

We analyze Elman-type Recurrent Reural Networks (RNNs) and their training in the mean-field regime. Specifically, we show convergence of gradient descent training dynamics of the RNN to the corresponding mean-field formulation in the large width limit. We also show that the fixed points of the limiting infinite-width dynamics are globally optimal, under some assumptions on the initialization of the weights. Our results establish optimality for feature-learning with wide RNNs in the mean-field regime