Comgra: A Tool for Analyzing and Debugging Neural Networks
This tool addresses the difficulty of debugging and analyzing neural networks for researchers and practitioners, but it is incremental as it builds on existing inspection methods.
The authors tackled the problem of inspecting neural networks by introducing Comgra, an open-source Python library for PyTorch that extracts internal activations and organizes them in a GUI, enabling debugging, architecture design, and interpretability without rerunning models.
Neural Networks are notoriously difficult to inspect. We introduce comgra, an open source python library for use with PyTorch. Comgra extracts data about the internal activations of a model and organizes it in a GUI (graphical user interface). It can show both summary statistics and individual data points, compare early and late stages of training, focus on individual samples of interest, and visualize the flow of the gradient through the network. This makes it possible to inspect the model's behavior from many different angles and save time by rapidly testing different hypotheses without having to rerun it. Comgra has applications for debugging, neural architecture design, and mechanistic interpretability. We publish our library through Python Package Index (PyPI) and provide code, documentation, and tutorials at https://github.com/FlorianDietz/comgra.