Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer
This addresses privacy concerns in computer vision by enabling effective use of encrypted images without sacrificing performance, though it appears incremental as it builds on existing domain adaptation and fine-tuning techniques.
The paper tackles the problem of performance degradation in privacy-preserving deep neural networks when using encrypted images, proposing a method that efficiently fine-tunes Vision Transformers with domain adaptation to avoid this issue. The result is improved classification accuracy on CIFAR-10 and ImageNet datasets compared to conventional methods.
We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT). The method allows us not only to train models and test with visually protected images but to also avoid the performance degradation caused from the use of encrypted images, whereas conventional methods cannot avoid the influence of image encryption. A domain adaptation method is used to efficiently fine-tune ViT with encrypted images. In experiments, the method is demonstrated to outperform conventional methods in an image classification task on the CIFAR-10 and ImageNet datasets in terms of classification accuracy.