Recurrent Neural Operators: Stable Long-Term PDE Prediction
This addresses a critical issue for researchers and practitioners using neural operators for PDE prediction, though it is an incremental improvement over existing methods.
The paper tackled the problem of error accumulation in long-term autoregressive predictions of time-dependent partial differential equations (PDEs) with neural operators, by proposing Recurrent Neural Operators (RNOs) that align training with inference dynamics, resulting in significantly improved accuracy and stability on standard benchmarks.
Neural operators have emerged as powerful tools for learning solution operators of partial differential equations. However, in time-dependent problems, standard training strategies such as teacher forcing introduce a mismatch between training and inference, leading to compounding errors in long-term autoregressive predictions. To address this issue, we propose Recurrent Neural Operators (RNOs)-a novel framework that integrates recurrent training into neural operator architectures. Instead of conditioning each training step on ground-truth inputs, RNOs recursively apply the operator to their own predictions over a temporal window, effectively simulating inference-time dynamics during training. This alignment mitigates exposure bias and enhances robustness to error accumulation. Theoretically, we show that recurrent training can reduce the worst-case exponential error growth typical of teacher forcing to linear growth. Empirically, we demonstrate that recurrently trained Multigrid Neural Operators significantly outperform their teacher-forced counterparts in long-term accuracy and stability on standard benchmarks. Our results underscore the importance of aligning training with inference dynamics for robust temporal generalization in neural operator learning.