Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training
This tool helps deep learning developers, who may lack system expertise, by offering an incremental improvement in performance profiling during DNN training.
The paper tackles the problem of debugging computational performance in deep neural network training by introducing Skyline, an interactive in-editor tool that provides performance predictions and visualizations, with a user study showing all participants found it useful and easy to use.
Training a state-of-the-art deep neural network (DNN) is a computationally-expensive and time-consuming process, which incentivizes deep learning developers to debug their DNNs for computational performance. However, effectively performing this debugging requires intimate knowledge about the underlying software and hardware systems---something that the typical deep learning developer may not have. To help bridge this gap, we present Skyline: a new interactive tool for DNN training that supports in-editor computational performance profiling, visualization, and debugging. Skyline's key contribution is that it leverages special computational properties of DNN training to provide (i) interactive performance predictions and visualizations, and (ii) directly manipulatable visualizations that, when dragged, mutate the batch size in the code. As an in-editor tool, Skyline allows users to leverage these diagnostic features to debug the performance of their DNNs during development. An exploratory qualitative user study of Skyline produced promising results; all the participants found Skyline to be useful and easy to use.