RTF-Q: Efficient Unsupervised Domain Adaptation with Retraining-free Quantization
This work addresses the problem of efficient unsupervised domain adaptation for edge devices, offering an incremental improvement by reducing training costs and resource usage.
The paper tackles the challenge of performing unsupervised domain adaptation on resource-constrained edge devices by proposing RTF-Q, a retraining-free quantization method that uses low-precision architectures and weight-sharing to adapt to dynamic computation budgets, achieving competitive accuracy with state-of-the-art methods while significantly reducing memory and computational costs.
Performing unsupervised domain adaptation on resource-constrained edge devices is challenging. Existing research typically adopts architecture optimization (e.g., designing slimmable networks) but requires expensive training costs. Moreover, it does not consider the considerable precision redundancy of parameters and activations. To address these limitations, we propose efficient unsupervised domain adaptation with ReTraining-Free Quantization (RTF-Q). Our approach uses low-precision quantization architectures with varying computational costs, adapting to devices with dynamic computation budgets. We subtly configure subnet dimensions and leverage weight-sharing to optimize multiple architectures within a single set of weights, enabling the use of pre-trained models from open-source repositories. Additionally, we introduce multi-bitwidth joint training and the SandwichQ rule, both of which are effective in handling multiple quantization bit-widths across subnets. Experimental results demonstrate that our network achieves competitive accuracy with state-of-the-art methods across three benchmarks while significantly reducing memory and computational costs.