Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matrices
This addresses privacy concerns in computer vision applications, but appears incremental as it builds on existing encryption methods for vision transformers.
The paper tackles the problem of performance degradation in privacy-preserving fine-tuning of vision transformers when using encrypted images, and proposes a new encryption method that achieves higher performance than conventional approaches.
We propose a novel method for privacy-preserving fine-tuning vision transformers (ViTs) with encrypted images. Conventional methods using encrypted images degrade model performance compared with that of using plain images due to the influence of image encryption. In contrast, the proposed encryption method using restricted random permutation matrices can provide a higher performance than the conventional ones.