Revisiting Long-Tailed Learning: Insights from an Architectural Perspective
It addresses the problem of designing effective neural architectures for long-tailed learning, which is crucial for real-world applications with imbalanced data, but it is incremental as it builds on existing methods.
This paper tackles the challenge of imbalanced data distributions in long-tailed recognition by analyzing how neural network architectures affect performance, and it proposes LT-DARTS, a neural architecture search method that achieves state-of-the-art results across multiple datasets with parameter efficiency.
Long-Tailed (LT) recognition has been widely studied to tackle the challenge of imbalanced data distributions in real-world applications. However, the design of neural architectures for LT settings has received limited attention, despite evidence showing that architecture choices can substantially affect performance. This paper aims to bridge the gap between LT challenges and neural network design by providing an in-depth analysis of how various architectures influence LT performance. Specifically, we systematically examine the effects of key network components on LT handling, such as topology, convolutions, and activation functions. Based on these observations, we propose two convolutional operations optimized for improved performance. Recognizing that operation interactions are also crucial to network effectiveness, we apply Neural Architecture Search (NAS) to facilitate efficient exploration. We propose LT-DARTS, a NAS method with a novel search space and search strategy specifically designed for LT data. Experimental results demonstrate that our approach consistently outperforms existing architectures across multiple LT datasets, achieving parameter-efficient, state-of-the-art results when integrated with current LT methods.