Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles
This work addresses the costly trial-and-error process in neural architecture design for AI researchers, but it is incremental as it builds on existing methods for architecture analysis.
The authors tackled the problem of understanding neural architecture spaces by developing ArchExplorer, a visual analysis method that reduces time complexity from O(kn^2N) to O(knN) and demonstrates effectiveness in summarizing design principles and selecting better-performing architectures through case studies.
Recent advances in artificial intelligence largely benefit from better neural network architectures. These architectures are a product of a costly process of trial-and-error. To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles. The key idea behind our method is to make the architecture space explainable by exploiting structural distances between architectures. We formulate the pairwise distance calculation as solving an all-pairs shortest path problem. To improve efficiency, we decompose this problem into a set of single-source shortest path problems. The time complexity is reduced from O(kn^2N) to O(knN). Architectures are hierarchically clustered according to the distances between them. A circle-packing-based architecture visualization has been developed to convey both the global relationships between clusters and local neighborhoods of the architectures in each cluster. Two case studies and a post-analysis are presented to demonstrate the effectiveness of ArchExplorer in summarizing design principles and selecting better-performing architectures.