In Situ Training of Implicit Neural Compressors for Scientific Simulations via Sketch-Based Regularization
This addresses the challenge of efficient data compression for complex scientific simulations, though it appears to be an incremental improvement on existing implicit neural representation methods.
The paper tackles the problem of catastrophic forgetting during in situ training of implicit neural compressors for scientific simulations by introducing a sketch-based regularization method, achieving strong reconstruction performance at high compression rates and matching offline method performance.
Focusing on implicit neural representations, we present a novel in situ training protocol that employs limited memory buffers of full and sketched data samples, where the sketched data are leveraged to prevent catastrophic forgetting. The theoretical motivation for our use of sketching as a regularizer is presented via a simple Johnson-Lindenstrauss-informed result. While our methods may be of wider interest in the field of continual learning, we specifically target in situ neural compression using implicit neural representation-based hypernetworks. We evaluate our method on a variety of complex simulation data in two and three dimensions, over long time horizons, and across unstructured grids and non-Cartesian geometries. On these tasks, we show strong reconstruction performance at high compression rates. Most importantly, we demonstrate that sketching enables the presented in situ scheme to approximately match the performance of the equivalent offline method.