An Extensible Benchmark Suite for Learning to Simulate Physical Systems
This provides a standardized foundation for evaluating and comparing data-driven methods in scientific computing, addressing fragmentation in the field.
The authors tackled the lack of unified benchmarks for data-driven simulation of physical systems by introducing an extensible benchmark suite with four representative systems and evaluation protocols, enabling objective assessment of stability, accuracy, and efficiency across methods like kernel-based, MLP, CNN, and nearest neighbors.
Simulating physical systems is a core component of scientific computing, encompassing a wide range of physical domains and applications. Recently, there has been a surge in data-driven methods to complement traditional numerical simulations methods, motivated by the opportunity to reduce computational costs and/or learn new physical models leveraging access to large collections of data. However, the diversity of problem settings and applications has led to a plethora of approaches, each one evaluated on a different setup and with different evaluation metrics. We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time integrators and representative data-driven methods (kernel-based, MLP, CNN, nearest neighbors). Our framework allows evaluating objectively and systematically the stability, accuracy, and computational efficiency of data-driven methods. Additionally, it is configurable to permit adjustments for accommodating other learning tasks and for establishing a foundation for future developments in machine learning for scientific computing.