Fundamentals of Regression
It offers a foundational overview for researchers and practitioners in scientific machine learning, but is incremental as it synthesizes existing concepts.
This chapter reviews regression in machine learning, transitioning from data-driven statistical approaches to physics-informed methods that integrate physical knowledge, and provides an overview of traditional techniques and their integration with computational science.
This chapter opens with a review of classic tools for regression, a subset of machine learning that seeks to find relationships between variables. With the advent of scientific machine learning this field has moved from a purely data-driven (statistical) formalism to a constrained or ``physics-informed'' formalism, which integrates physical knowledge and methods from traditional computational engineering. In the first part, we introduce the general concepts and the statistical flavor of regression versus other forms of curve fitting. We then move to an overview of traditional methods from machine learning and their classification and ways to link these to traditional computational science. Finally, we close with a note on methods to combine machine learning and numerical methods for physics