Perspective: Energy Landscapes for Machine Learning
This work provides an interdisciplinary perspective for researchers in machine learning and physical sciences, offering tools to analyze complex models, but it is incremental as it adapts existing methods to a new context.
The paper tackles the challenge of analyzing machine learning models with multiple local minima by applying methods from molecular energy landscapes to visualize and understand the solution space, resulting in new insights into training dynamics and prediction properties.
Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular potential energy landscapes can be applied to these machine learning landscapes to gain new insight into the solution space involved in training and the nature of the corresponding predictions. In particular, we can define quantities analogous to molecular structure, thermodynamics, and kinetics, and relate these emergent properties to the structure of the underlying landscape. This Perspective aims to describe these analogies with examples from recent applications, and suggest avenues for new interdisciplinary research.