Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems
This addresses convergence issues in physics-based inverse problems for researchers and engineers, though it is incremental as it builds on existing neural network and optimization methods.
The paper tackled the problem of iterative optimization algorithms getting stuck in local minima or chaotic regions when solving physics-based inverse problems, by using deep neural networks to replicate and smooth the loss landscapes, resulting in improved convergence in predicting optimum inverse parameters compared to conventional optimizers like BFGS.
Solving inverse problems in physics is central to understanding complex systems and advancing technologies in various fields. Iterative optimization algorithms, commonly used to solve these problems, often encounter local minima, chaos, or regions with zero gradients. This is due to their overreliance on local information and highly chaotic inverse loss landscapes governed by underlying partial differential equations (PDEs). In this work, we show that deep neural networks successfully replicate such complex loss landscapes through spatio-temporal trajectory inputs. They also offer the potential to control the underlying complexity of these chaotic loss landscapes during training through various regularization methods. We show that optimizing on network-smoothened loss landscapes leads to improved convergence in predicting optimum inverse parameters over conventional momentum-based optimizers such as BFGS on multiple challenging problems.