Hadaptive-Net: Efficient Vision Models via Adaptive Cross-Hadamard Synergy
This work addresses the need for efficient vision models for applications like mobile or edge computing, though it appears incremental as it builds on existing Hadamard product concepts.
The paper tackled the underutilization of Hadamard products in vision models by proposing Hadaptive-Net, a lightweight backbone that achieves a balance between inference speed and accuracy, with experiments showing competitive performance on visual tasks.
Recent studies have revealed the immense potential of Hadamard product in enhancing network representational capacity and dimensional compression. However, despite its theoretical promise, this technique has not been systematically explored or effectively applied in practice, leaving its full capabilities underdeveloped. In this work, we first analyze and identify the advantages of Hadamard product over standard convolutional operations in cross-channel interaction and channel expansion. Building upon these insights, we propose a computationally efficient module: Adaptive Cross-Hadamard (ACH), which leverages adaptive cross-channel Hadamard products for high-dimensional channel expansion. Furthermore, we introduce Hadaptive-Net (Hadamard Adaptive Network), a lightweight network backbone for visual tasks, which is demonstrated through experiments that it achieves an unprecedented balance between inference speed and accuracy through our proposed module.