PEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning
This addresses the challenge of hyperparameter tuning in preconditioner design for iterative solvers, though it appears incremental as an enhancement to existing neural preconditioner methods.
The paper tackles the problem of learning matrix preconditioners by introducing PEARL, a reinforcement learning approach using an actor-critic model with a dual-objective function, which demonstrates improved flexibility and iterative solving speed compared to traditional and neural preconditioners.
We present PEARL (Preconditioner Enhancement through Actor-critic Reinforcement Learning), a novel approach to learning matrix preconditioners. Existing preconditioners such as Jacobi, Incomplete LU, and Algebraic Multigrid methods offer problem-specific advantages but rely heavily on hyperparameter tuning. Recent advances have explored using deep neural networks to learn preconditioners, though challenges such as misbehaved objective functions and costly training procedures remain. PEARL introduces a reinforcement learning approach for learning preconditioners, specifically, a contextual bandit formulation. The framework utilizes an actor-critic model, where the actor generates the incomplete Cholesky decomposition of preconditioners, and the critic evaluates them based on reward-specific feedback. To further guide the training, we design a dual-objective function, combining updates from the critic and condition number. PEARL contributes a generalizable preconditioner learning method, dynamic sparsity exploration, and cosine schedulers for improved stability and exploratory power. We compare our approach to traditional and neural preconditioners, demonstrating improved flexibility and iterative solving speed.