Residual-Quantile Adjustment for Adaptive Training of Physics-informed Neural Network
This work addresses the problem of improving training efficiency for PINNs in solving PDEs, which is incremental as it builds on prior adaptive methods by refining weight selection based on residual quantiles.
The paper tackles the challenge of efficiently selecting optimal weight distributions for adaptive training in Physics-informed Neural Networks (PINNs) by focusing on the tail behavior of numerical residuals, proposing the Residual-Quantile Adjustment (RQA) method that reassigns weights above a quantile to the median. The results show that RQA outperforms several existing adaptive methods on various PDE problems.
Adaptive training methods for Physics-informed neural network (PINN) require dedicated constructions of the distribution of weights assigned at each training sample. To efficiently seek such an optimal weight distribution is not a simple task and most existing methods choose the adaptive weights based on approximating the full distribution or the maximum of residuals. In this paper, we show that the bottleneck in the adaptive choice of samples for training efficiency is the behavior of the tail distribution of the numerical residual. Thus, we propose the Residual-Quantile Adjustment (RQA) method for a better weight choice for each training sample. After initially setting the weights proportional to the $p$-th power of the residual, our RQA method reassign all weights above $q$-quantile ($90\%$ for example) to the median value, so that the weight follows a quantile-adjusted distribution derived from the residuals. This iterative reweighting technique, on the other hand, is also very easy to implement. Experiment results show that the proposed method can outperform several adaptive methods on various partial differential equation (PDE) problems.