Continual learning autoencoder training for a particle-in-cell simulation via streaming
This addresses the challenge of data storage and training efficiency for physics simulations in the exascale era, though it is incremental as it adapts existing continual learning methods to a specific domain.
The authors tackled the problem of training machine learning models for high-resolution physics simulations in the exascale era, where storing data on disk is infeasible, by developing a pipeline that trains a 3D autoencoder concurrently with a running particle-in-cell simulation using in-memory streaming and continual learning methods, achieving enhanced generalization without disk storage.
The upcoming exascale era will provide a new generation of physics simulations. These simulations will have a high spatiotemporal resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. Therefore, we need to rethink the training of machine learning models for simulations for the upcoming exascale era. This work presents an approach that trains a neural network concurrently to a running simulation without storing data on a disk. The training pipeline accesses the training data by in-memory streaming. Furthermore, we apply methods from the domain of continual learning to enhance the generalization of the model. We tested our pipeline on the training of a 3d autoencoder trained concurrently to laser wakefield acceleration particle-in-cell simulation. Furthermore, we experimented with various continual learning methods and their effect on the generalization.