AutoBSS: An Efficient Algorithm for Block Stacking Style Search
This addresses the need for better neural network architecture design by automating BSS search, offering broad improvements across tasks like classification, detection, and segmentation, though it is incremental in focusing on a specific configuration aspect.
The paper tackles the problem of optimizing Block Stacking Style (BSS) in neural networks, which is often overlooked, and proposes AutoBSS, an efficient AutoML algorithm that achieves significant performance gains, such as improving ResNet50 on ImageNet to 79.29% accuracy.
Neural network architecture design mostly focuses on the new convolutional operator or special topological structure of network block, little attention is drawn to the configuration of stacking each block, called Block Stacking Style (BSS). Recent studies show that BSS may also have an unneglectable impact on networks, thus we design an efficient algorithm to search it automatically. The proposed method, AutoBSS, is a novel AutoML algorithm based on Bayesian optimization by iteratively refining and clustering Block Stacking Style Code (BSSC), which can find optimal BSS in a few trials without biased evaluation. On ImageNet classification task, ResNet50/MobileNetV2/EfficientNet-B0 with our searched BSS achieve 79.29%/74.5%/77.79%, which outperform the original baselines by a large margin. More importantly, experimental results on model compression, object detection and instance segmentation show the strong generalizability of the proposed AutoBSS, and further verify the unneglectable impact of BSS on neural networks.