Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
This provides a theoretical foundation for physics-informed models in real-world nonlinear applications, though it appears incremental by extending analysis to hybrid settings.
The paper tackles the problem of understanding generalization in physics-informed machine learning for hybrid settings with incomplete physical constraints, establishing that generalization performance is governed by the dimension of the affine variety associated with the constraint rather than parameter count, with experimental validation supporting this theoretical finding.
Physics-informed machine learning is gaining significant traction for enhancing statistical performance and sample efficiency through the integration of physical knowledge. However, current theoretical analyses often presume complete prior knowledge in non-hybrid settings, overlooking the crucial integration of observational data, and are frequently limited to linear systems, unlike the prevalent nonlinear nature of many real-world applications. To address these limitations, we introduce a unified residual form that unifies collocation and variational methods, enabling the incorporation of incomplete and complex physical constraints in hybrid learning settings. Within this formulation, we establish that the generalization performance of physics-informed regression in such hybrid settings is governed by the dimension of the affine variety associated with the physical constraint, rather than by the number of parameters. This enables a unified analysis that is applicable to both linear and nonlinear equations. We also present a method to approximate this dimension and provide experimental validation of our theoretical findings.