Neuro-Visualizer: An Auto-encoder-based Loss Landscape Visualization Method
This addresses the need for more flexible and high-fidelity visualization tools for researchers studying neural networks, though it appears incremental as it builds on existing non-linear approaches.
The paper tackles the problem of visualizing neural network loss landscapes by introducing Neuro-Visualizer, an auto-encoder-based non-linear method that outperforms linear and non-linear baselines in experiments on knowledge-guided machine learning applications.
In recent years, there has been a growing interest in visualizing the loss landscape of neural networks. Linear landscape visualization methods, such as principal component analysis, have become widely used as they intuitively help researchers study neural networks and their training process. However, these linear methods suffer from limitations and drawbacks due to their lack of flexibility and low fidelity at representing the high dimensional landscape. In this paper, we present a novel auto-encoder-based non-linear landscape visualization method called Neuro-Visualizer that addresses these shortcoming and provides useful insights about neural network loss landscapes. To demonstrate its potential, we run experiments on a variety of problems in two separate applications of knowledge-guided machine learning (KGML). Our findings show that Neuro-Visualizer outperforms other linear and non-linear baselines and helps corroborate, and sometime challenge, claims proposed by machine learning community. All code and data used in the experiments of this paper are available at an anonymous link https://anonymous.4open.science/r/NeuroVisualizer-FDD6