GINN: Geometric Illustration of Neural Networks
This is an incremental contribution providing a visualization tool for researchers studying neural network behavior, but it lacks broad impact or novelty.
The authors tackled the problem of visualizing decision boundaries in neural networks by developing the GINN tool to track ReLU unit activations during training on a pixel intensity prediction task, resulting in observations of several phenomena without specific numerical results.
This informal technical report details the geometric illustration of decision boundaries for ReLU units in a three layer fully connected neural network. The network is designed and trained to predict pixel intensity from an (x, y) input location. The Geometric Illustration of Neural Networks (GINN) tool was built to visualise and track the points at which ReLU units switch from being active to off (or vice versa) as the network undergoes training. Several phenomenon were observed and are discussed herein. This technical report is a supporting document to the blog post with online demos and is available at http://www.bayeswatch.com/2018/09/17/GINN/.