Shoko Imaizumi

Hiroto Sawada, Shoko Imaizumi, Hitoshi Kiya

In this paper, we propose a method for privacy-preserving federated learning that uses randomly selected model parameters to update global models. High-quality deep neural networks (DNN) models require a huge amount of training data in general, but model training raises privacy concerns when dealing with sensitive or personal information. Federated learning is a distributed machine learning framework in which multiple clients and a server train a model collaboratively. However, if the shared updates are compromised, an attacker may reconstruct the original training data. In addition, previous methods for improving robustness generally reduce the accuracy. To overcome these issues, in our method called federated learning using randomly selected model parameters (FLRSP), model parameters computed in each local server are randomly selected and shared to update a global model in a central server. In experiments, image classification tasks were carried out on the ResNet34 architecture and the Vision Transformer (ViT) under the use of Federated Stochastic Gradient Descent (FedSGD) and Federated Averaging (FedAvg), and the results demonstrated our method's effectiveness in terms of image classification accuracy and robustness against state-of-the-art attacks compared with previous methods.

Mare Hirose, Shoko Imaizumi, Hitoshi Kiya

This paper proposes a novel privacy-preserving semantic segmentation method that can use independent keys for each client and image. In the proposed method, the model creator and each client encrypt images using locally generated keys, and model training and inference are conducted on the encrypted images. To mitigate performance degradation, an image encryption method is applied to model training in addition to the generation of test images. In experiments, the effectiveness of the proposed method is confirmed on the Cityscapes dataset under the use of a vision transformer-based model, called SETR.

Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya

Privacy-preserving action recognition (PPAR) enables machines to understand human activities in videos without revealing sensitive visual content. Among the various strategies for PPAR, encryption-based methods achieve strong privacy protection while maintaining high recognition performance. However, these methods lead to a catastrophic decrease in recognition performance and visual quality when the encrypted videos are compressed. That is, the previous methods are not compression-friendly. To address these issues, in this paper, we propose the first compression-friendly encryption method for PPAR, called CFE-PPAR. In CFE-PPAR, videos encrypted with secret keys can be directly recognized by a video transformer, which uses parameters transformed by the same keys as those used for video encryption. In experiments, it is verified that CFE-PPAR outperforms previous methods on the UCF101 and HMDB51 datasets under Motion-JPEG and H.264 compression.

Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya

We propose a low-rank adaptation method for training privacy-preserving vision transformer (ViT) models that efficiently freezes pre-trained ViT model weights. In the proposed method, trainable rank decomposition matrices are injected into each layer of the ViT architecture, and moreover, the patch embedding layer is not frozen, unlike in the case of the conventional low-rank adaptation methods. The proposed method allows us not only to reduce the number of trainable parameters but to also maintain almost the same accuracy as that of full-time tuning.

Hitoshi Kiya, AprilPyone MaungMaung, Yuma Kinoshita et al.

This article presents an overview of image transformation with a secret key and its applications. Image transformation with a secret key enables us not only to protect visual information on plain images but also to embed unique features controlled with a key into images. In addition, numerous encryption methods can generate encrypted images that are compressible and learnable for machine learning. Various applications of such transformation have been developed by using these properties. In this paper, we focus on a class of image transformation referred to as learnable image encryption, which is applicable to privacy-preserving machine learning and adversarially robust defense. Detailed descriptions of both transformation algorithms and performances are provided. Moreover, we discuss robustness against various attacks.

Kotoko Hiraoka, Kensuke Fukumoto, Takashi Yamazoe et al.

We propose an efficient framework with compatibility between normal printing and printing with special color inks in this paper. Special color inks can be used for printing to represent some particular colors and specific optical properties, which are difficult to express using only CMYK inks. Special color layers are required in addition to the general color layer for printing with special color inks. We introduce a reversible data hiding (RDH) method to embed the special color layers into the general color layer without visible artifacts. The proposed method can realize both normal printing and printing with special color inks by using a single layer. Our experimental results show that the quality of the marked image is virtually identical to that of the original image, i.e., the general color layer.

Minagi Ueda, Shoko Imaizumi

We propose an efficient framework of reversible data hiding to preserve compatibility between normal printing and printing with a special color ink by using a single common image. The special color layer is converted to a binary image by digital halftoning and losslessly compressed using JBIG2. Then, the compressed information of the binarized special color layer is reversibly embedded into the general color layer without significant distortion. Our experimental results show the availability of the proposed method in terms of the marked image quality.

Shoko Imaizumi, Yusuke Izawa, Ryoichi Hirasawa et al.

We propose a reversible data hiding (RDH) method in compressible encrypted images called the encryption-then-compression (EtC) images. The proposed method allows us to not only embed a payload in encrypted images but also compress the encrypted images containing the payload. In addition, the proposed RDH method can be applied to both plain images and encrypted ones, and the payload can be extracted flexibly in the encrypted domain or from the decrypted images. Various RDH methods have been studied in the encrypted domain, but they are not considered to be two-domain data hiding, and the resultant images cannot be compressed by using image coding standards, such as JPEG-LS and JPEG 2000. In our experiment, the proposed method shows high performance in terms of lossless compression efficiency by using JPEG-LS and JPEG 2000, data hiding capacity, and marked image quality.

Shoko Imaizumi, Hitoshi Kiya

This paper proposes a block-permutation-based encryption (BPBE) scheme for the encryption-then-compression (ETC) system that enhances the color scrambling. A BPBE image can be obtained through four processes, positional scrambling, block rotation/flip, negative-positive transformation, and color component shuffling, after dividing the original image into multiple blocks. The proposed scheme scrambles the R, G, and B components independently in positional scrambling, block rotation/flip, and negative-positive transformation, by assigning different keys to each color component. The conventional scheme considers the compression efficiency using JPEG and JPEG 2000, which need a color conversion before the compression process by default. Therefore, the conventional scheme scrambles the color components identically in each process. In contrast, the proposed scheme takes into account the RGB-based compression, such as JPEG-LS, and thus can increase the extent of the scrambling. The resilience against jigsaw puzzle solver (JPS) can consequently be increased owing to the wider color distribution of the BPBE image. Additionally, the key space for resilience against brute-force attacks has also been expanded exponentially. Furthermore, the proposed scheme can maintain the JPEG-LS compression efficiency compared to the conventional scheme. We confirm the effectiveness of the proposed scheme by experiments and analyses.

Warit Sirichotedumrong, Tatsuya Chuman, Shoko Imaizumi et al.

This paper proposes a new block scrambling encryption scheme that enhances the security of encryption-then-compression (EtC) systems for JPEG images, which are used, for example, to securely transmit images through an untrusted channel provider. The proposed method allows the use of a smaller block size and a larger number of blocks than the conventional ones. Moreover, images encrypted using proposed scheme include less color information due to the use of grayscale even when the original image has three color channels. These features enhance security against various attacks such as jigsaw puzzle solver and brute-force attacks. The results of an experiment in which encrypted images were uploaded to and then downloaded from Twitter and Facebook demonstrated the effectiveness of the proposed scheme for EtC systems.