How Much Memory Do We Need? Adaptive Memory Gate for Neural Operators
For researchers using neural operators for time-dependent PDEs, this work addresses a key limitation of memory-augmented models by enabling adaptive memory weighting, though the improvement is demonstrated on two specific equations.
The paper tackles the problem of fixed memory weights in neural operators for PDE solving, which limits adaptability across different resolutions and physical parameters. The proposed AMGFNO achieves 55-79% nRMSE reduction on Kuramoto-Sivashinsky and Burgers' equations at low resolution, with the learned gate automatically adjusting from ~0.7 to near-zero as resolution increases.
Neural operators have emerged as a powerful data-driven approach for solving time-dependent PDEs. Among recent advances, memory-augmented neural operators explicitly incorporate past states and have achieved remarkable performance under low-resolution observation settings. However, existing approaches apply a fixed memory weight regardless of observation conditions, such as resolution or physical parameters, limiting their adaptability. Our preliminary experiments reveal that optimal memory weight varies with resolution and viscosity, implying that a fixed memory weight cannot simultaneously optimize performance across diverse settings. We propose AMGFNO, which dynamically modulates memory weight through a learnable gate. On the Kuramoto-Sivashinsky and Burgers' equations, AMGFNO achieves 55-79% nRMSE reduction over at low resolution, with the learned gate value automatically decreasing from $\bar{g} \approx 0.7$ to near-zero as resolution increases.