Rethinking Differentiable Search for Mixed-Precision Neural Networks
This addresses the need for efficient inference on edge devices by optimizing bit-width allocation per filter, though it is an incremental improvement over existing mixed-precision methods.
The paper tackled the problem of suboptimal uniform bit-width in low-precision neural networks by proposing a differentiable search method for mixed-precision networks, resulting in learned networks that significantly outperform uniform counterparts, such as efficiently searching Inception-V3 on ImageNet without a proxy task.
Low-precision networks, with weights and activations quantized to low bit-width, are widely used to accelerate inference on edge devices. However, current solutions are uniform, using identical bit-width for all filters. This fails to account for the different sensitivities of different filters and is suboptimal. Mixed-precision networks address this problem, by tuning the bit-width to individual filter requirements. In this work, the problem of optimal mixed-precision network search (MPS) is considered. To circumvent its difficulties of discrete search space and combinatorial optimization, a new differentiable search architecture is proposed, with several novel contributions to advance the efficiency by leveraging the unique properties of the MPS problem. The resulting Efficient differentiable MIxed-Precision network Search (EdMIPS) method is effective at finding the optimal bit allocation for multiple popular networks, and can search a large model, e.g. Inception-V3, directly on ImageNet without proxy task in a reasonable amount of time. The learned mixed-precision networks significantly outperform their uniform counterparts.