Yunhui He

Yunhui He, Scott P. MacLachlan

In this paper, we develop a local Fourier analysis of multigrid methods based on block-structured relaxation schemes for stable and stabilized mixed finite-element discretizations of the Stokes equations, to analyze their convergence behavior. Three relaxation schemes are considered: distributive, Braess-Sarazin, and Uzawa relaxation. From this analysis, parameters that minimize the local Fourier analysis smoothing factor are proposed for the stabilized methods with distributive and Braess-Sarazin relaxation. Considering the failure of the local Fourier analysis smoothing factor in predicting the true two-grid convergence factor for the stable discretization, we numerically optimize the two-grid convergence predicted by local Fourier analysis in this case. We also compare the efficiency of the presented algorithms with variants using inexact solvers. Finally, some numerical experiments are presented to validate the two-grid and multigrid convergence factors.

Yunhui He, Leo G Rebholz

We consider nonlinear GMRES (NGMRES) as an acceleration technique for the Navier-Stokes Picard iteration, a direction that has not previously been explored. We identify the optimal norm for the least squares optimization problem arising in the NGMRES algorithm, and establish a convergence analysis for NGMRES with general depth that proves NGMRES scales the Picard Lipschitz constant by the gain of the optimization problem. To our knowledge, this is the first convergence proof for NGMRES that identifies the mechanism responsible for convergence acceleration. Numerical experiments demonstrate that the convergence estimates are remarkably sharp. In addition, NGMRES greatly improves the performance of the Picard iteration, even in cases where the unaccelerated iteration diverges.

Yunhui He, Hehu Xie

In this paper, we consider the shift-inverse method with Richardson iteration step for the eigenvalue problems. It will be shown that the convergence speed depends heavily on the eigenvalue gap between the desired eigenvalue and undesired ones.

Yunhui He, Yu Li, Hehu Xie

We propose a new type of multilevel method for solving eigenvalue problems based on Newton iteration. With the proposed iteration method, solving eigenvalue problem on the finest finite element space is replaced by solving a small scale eigenvalue problem in a coarse space and solving a series of augmented linear problems, derived by Newton step in the corresponding series of finite element spaces. This iteration scheme improves overall efficiency of the finite element method for solving eigenvalue problems. Finally, some numerical examples are provided to validate the efficiency of the proposed numerical scheme.

Yunhui He, Leo Rebholz

In this work, we extend a modified Anderson acceleration proposed in [Y. He, arXiv:2603.25983, 2026] to accelerate the Picard iteration for the Navier-Stokes equations. In this variant of Anderson acceleration, named AAg, the nonlinear residual--rather than the standard fixed-point iteration residual--is used to define the associated least-squares problem. We establish a convergence analysis for this method with any depth that shows how AAg accelerates convergence through the gain of the optimization problem, and obtain a sharp prediction of its linear convergence rate (a feature that is not part of the known theory for classical Anderson acceleration). Additionally, motivated by this sharp convergence prediction, we introduce an adaptive strategy that automatically selects the depth parameter. Results of several numerical experiments are given that illustrate the new theory and also demonstrate the effectiveness of the proposed adaptive approach. Comparisons of AAg to usual AA and nonlinear GMRES are also provided.

Yunhui He

In this work, we propose new variants of Anderson acceleration and nonlinear GMRES for general fixed-point iterations, based on modified least-squares problems associated with the methods. To solve the underlying linear systems, we apply these new approaches to accelerate the preconditioned Richardson iteration. We establish connections between the proposed variants and both left- and right-preconditioned GMRES. In particular, we show that full NGMRES applied to the preconditioned Richardson iteration is equivalent to right-preconditioned GMRES, while full NGMRES equipped with the new least-squares formulation is equivalent to left-preconditioned GMRES. Furthermore, under certain conditions on the preconditioned coefficient matrix, an equivalence between windowed NGMRES with any depth and preconditioned GMRES. These theoretical results deepen our understanding of NGMRES for solving linear systems and clarify its relationship to classical preconditioned GMRES. Finally, we establish conditions for monotonicity of the various variants. Numerical results are presented to validate our theoretical findings.

Hans De Sterck, Yunhui He, Oliver A. Krzysik

Anderson acceleration (AA) is widely used for accelerating the convergence of nonlinear fixed-point methods $x_{k+1}=q(x_{k})$, $x_k \in \mathbb{R}^n$, but little is known about how to quantify the convergence acceleration provided by AA. As a roadway towards gaining more understanding of convergence acceleration by AA, we study AA($m$), i.e., Anderson acceleration with finite window size $m$, applied to the case of linear fixed-point iterations $x_{k+1}=M x_{k}+b$. We write AA($m$) as a Krylov method with polynomial residual update formulas, and derive recurrence relations for the AA($m$) polynomials. Writing AA($m$) as a Krylov method immediately implies that $k$ iterations of AA($m$) cannot produce a smaller residual than $k$ iterations of GMRES without restart (but without implying anything about the relative convergence speed of (windowed) AA($m$) versus restarted GMRES($m$)). We find that the AA($m$) residual polynomials observe a periodic memory effect where increasing powers of the error iteration matrix $M$ act on the initial residual as the iteration number increases. We derive several further results based on these polynomial residual update formulas, including orthogonality relations, a lower bound on the AA(1) acceleration coefficient $β_k$, and explicit nonlinear recursions for the AA(1) residuals and residual polynomials that do not include the acceleration coefficient $β_k$. Using these recurrence relations we also derive new residual convergence bounds for AA(1) in the linear case, demonstrating how the per-iteration residual reduction $||r_{k+1}||/||r_{k}||$ depends strongly on the residual reduction in the previous iteration and on the angle between the prior residual vectors $r_k$ and $r_{k-1}$. We apply these results to study the influence of the initial guess on the asymptotic convergence factor of AA(1), and to study AA(1) residual convergence patterns.

Hans De Sterck, Yunhui He

We study the asymptotic convergence of AA($m$), i.e., Anderson acceleration with window size $m$ for accelerating fixed-point methods $x_{k+1}=q(x_{k})$, $x_k \in R^n$. Convergence acceleration by AA($m$) has been widely observed but is not well understood. We consider the case where the fixed-point iteration function $q(x)$ is differentiable and the convergence of the fixed-point method itself is root-linear. We identify numerically several conspicuous properties of AA($m$) convergence: First, AA($m$) sequences $\{x_k\}$ converge root-linearly but the root-linear convergence factor depends strongly on the initial condition. Second, the AA($m$) acceleration coefficients $β^{(k)}$ do not converge but oscillate as $\{x_k\}$ converges to $x^*$. To shed light on these observations, we write the AA($m$) iteration as an augmented fixed-point iteration $z_{k+1} =Ψ(z_k)$, $z_k \in R^{n(m+1)}$ and analyze the continuity and differentiability properties of $Ψ(z)$ and $β(z)$. We find that the vector of acceleration coefficients $β(z)$ is not continuous at the fixed point $z^*$. However, we show that, despite the discontinuity of $β(z)$, the iteration function $Ψ(z)$ is Lipschitz continuous and directionally differentiable at $z^*$ for AA(1), and we generalize this to AA($m$) with $m>1$ for most cases. Furthermore, we find that $Ψ(z)$ is not differentiable at $z^*$. We then discuss how these theoretical findings relate to the observed convergence behaviour of AA($m$). The discontinuity of $β(z)$ at $z^*$ allows $β^{(k)}$ to oscillate as $\{x_k\}$ converges to $x^*$, and the non-differentiability of $Ψ(z)$ allows AA($m$) sequences to converge with root-linear convergence factors that strongly depend on the initial condition. Additional numerical results illustrate our findings.

Dawei Wang, Yunhui He, Hans De Sterck

Empirical results show that Anderson acceleration (AA) can be a powerful mechanism to improve the asymptotic linear convergence speed of the Alternating Direction Method of Multipliers (ADMM) when ADMM by itself converges linearly. However, theoretical results to quantify this improvement do not exist yet. In this paper we explain and quantify this improvement in linear asymptotic convergence speed for the special case of a stationary version of AA applied to ADMM. We do so by considering the spectral properties of the Jacobians of ADMM and the stationary version of AA evaluated at the fixed point, where the coefficients of the stationary AA method are computed such that its asymptotic linear convergence factor is optimal. The optimal linear convergence factors of this stationary AA-ADMM method are computed analytically or by optimization, based on previous work on optimal stationary AA acceleration. Using this spectral picture and those analytical results, our approach provides new insight into how and by how much the stationary AA method can improve the asymptotic linear convergence factor of ADMM. Numerical results also indicate that the optimal linear convergence factor of the stationary AA methods gives a useful estimate for the asymptotic linear convergence speed of the non-stationary AA method that is used in practice.

Hans De Sterck, Yunhui He

We consider nonlinear convergence acceleration methods for fixed-point iteration $x_{k+1}=q(x_k)$, including Anderson acceleration (AA), nonlinear GMRES (NGMRES), and Nesterov-type acceleration (corresponding to AA with window size one). We focus on fixed-point methods that converge asymptotically linearly with convergence factor $ρ<1$ and that solve an underlying fully smooth and non-convex optimization problem. It is often observed that AA and NGMRES substantially improve the asymptotic convergence behavior of the fixed-point iteration, but this improvement has not been quantified theoretically. We investigate this problem under simplified conditions. First, we consider stationary versions of AA and NGMRES, and determine coefficients that result in optimal asymptotic convergence factors, given knowledge of the spectrum of $q'(x)$ at the fixed point $x^*$. This allows us to understand and quantify the asymptotic convergence improvement that can be provided by nonlinear convergence acceleration, viewing $x_{k+1}=q(x_k)$ as a nonlinear preconditioner for AA and NGMRES. Second, for the case of infinite window size, we consider linear asymptotic convergence bounds for GMRES applied to the fixed-point iteration linearized about $x^*$. Since AA and NGMRES are equivalent to GMRES in the linear case, one may expect the GMRES convergence factors to be relevant for AA and NGMRES as $x_k \rightarrow x^*$. Our results are illustrated numerically for a class of test problems from canonical tensor decomposition, comparing steepest descent and alternating least squares (ALS) as the fixed-point iterations that are accelerated by AA and NGMRES. Our numerical tests show that both approaches allow us to estimate asymptotic convergence speed for nonstationary AA and NGMRES with finite window size.

Yunhui He, Qichen Hong, Hehu Xie et al.

This paper is to give a new understanding and applications of the subspace projection method for selfadjoint eigenvalue problems. A new error estimate in the energy norm, which is induced by the stiff matrix, of the subspace projection method for eigenvalue problems is given. The relation between error estimates in $L^2$-norm and energy norm is also deduced. Based on this relation, a new type of inverse power method is designed for eigenvalue problems and the corresponding convergence analysis is also provided. Then we present the analysis of the geometric and algebraic multigrid methods for eigenvalue problems based on the convergence result of the new inverse power method.

Hongtao Chen, Yunhui He, Yu Li et al.

A multigrid method is proposed in this paper to solve eigenvalue problems by the finite element method based on the shifted-inverse power iteration technique. With this scheme, solving eigenvalue problem is transformed to a series of nonsingular solutions of boundary value problems on multilevel meshes. Since replacing the difficult eigenvalue solving by the easier solution of boundary value problems, the multigrid way can improve the overall efficiency of the eigenvalue problem solving. Some numerical experiments are presented to validate the efficiency of this new method.