NeuralVis: Visualizing and Interpreting Deep Learning Models
This addresses the difficulty software engineers face in interpreting deep learning models, but it is incremental as it builds on existing visualization and adversarial attack methods.
The paper tackles the problem of analyzing and understanding deep neural network behaviors for software engineers by presenting NeuralVis, a visualization tool that assists in identifying critical features for predictions, as demonstrated in a user study with ten participants on LeNet and VGG-12 models.
Deep Neural Network(DNN) techniques have been prevalent in software engineering. They are employed to faciliatate various software engineering tasks and embedded into many software applications. However, analyzing and understanding their behaviors is a difficult task for software engineers. In this paper, to support software engineers in visualizing and interpreting deep learning models, we present NeuralVis, an instance-based visualization tool for DNN. NeuralVis is designed for: 1). visualizing the structure of DNN models, i.e., components, layers, as well as connections; 2). visualizing the data transformation process; 3). integrating existing adversarial attack algorithms for test input generation; 4). comparing intermediate outputs of different instances to guide the test input generation; To demonstrate the effectiveness of NeuralVis, we conduct an user study involving ten participants on two classic DNN models, i.e., LeNet and VGG-12. The result shows NeuralVis can assist developers in identifying the critical features that determines the prediction results. Video: https://youtu.be/hRxCovrOZFI