Understanding Neural Networks via Feature Visualization: A survey
This is an incremental survey that provides a comprehensive overview of feature visualization methods for researchers in interpretable AI and neuroscience.
The paper surveys Activation Maximization (AM) techniques for understanding neural networks by synthesizing stimuli that highly activate neurons, reviewing existing methods, a probabilistic interpretation, and applications in debugging and explaining networks.
A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine learning enable a family of methods to synthesize preferred stimuli that cause a neuron in an artificial or biological brain to fire strongly. Those methods are known as Activation Maximization (AM) or Feature Visualization via Optimization. In this chapter, we (1) review existing AM techniques in the literature; (2) discuss a probabilistic interpretation for AM; and (3) review the applications of AM in debugging and explaining networks.