Statistical Properties of Training & Generalization
For researchers applying deep learning to physics, this review clarifies how scaling laws and inductive biases affect model performance, but it is primarily a survey without new results.
This paper reviews neural scaling laws and their interplay with constraints and inductive biases in physics-informed deep learning, highlighting key features and surprises from a physics perspective.
Deep learning has managed to evade numerous intuitions from classical statistics to achieve unprecedented performance on a number of real-world tasks. In this article, we investigate the key features and surprises of deep learning from a physics-informed perspective, taking care to point out and justify where possible the many choices inherent in constructing a deep learning model. In particular, we review the phenomenon of neural scaling laws and discuss their interplay with the constraints and inductive biases which may be present when applying machine learning to problems in physics.