Yan Ke

Xiaoyu Bai, Fan Bai, Xiaofei Huo et al.

Radiotherapists require accurate registration of MR/CT images to effectively use information from both modalities. In a typical registration pipeline, rigid or affine transformations are applied to roughly align the fixed and moving images before proceeding with the deformation step. While recent learning-based methods have shown promising results in the rigid/affine step, these methods often require images with similar field-of-view (FOV) for successful alignment. As a result, aligning images with different FOVs remains a challenging task. Self-supervised landmark detection methods like self-supervised Anatomical eMbedding (SAM) have emerged as a useful tool for mapping and cropping images to similar FOVs. However, these methods are currently limited to intra-modality use only. To address this limitation and enable cross-modality matching, we propose a new approach called Cross-SAM. Our approach utilizes a novel iterative process that alternates between embedding learning and CT-MRI registration. We start by applying aggressive contrast augmentation on both CT and MRI images to train a SAM model. We then use this SAM to identify corresponding regions on paired images using robust grid-points matching, followed by a point-set based affine/rigid registration, and a deformable fine-tuning step to produce registered paired images. We use these registered pairs to enhance the matching ability of SAM, which is then processed iteratively. We use the final model for cross-modality matching tasks. We evaluated our approach on two CT-MRI affine registration datasets and found that Cross-SAM achieved robust affine registration on both datasets, significantly outperforming other methods and achieving state-of-the-art performance.

Ke Yan, Qing Cai, Fan Zhang et al.

Although semi-supervised learning has made significant advances in the field of medical image segmentation, fully annotating a volumetric sample slice by slice remains a costly and time-consuming task. Even worse, most of the existing approaches pay much attention to image-level information and ignore semantic features, resulting in the inability to perceive weak boundaries. To address these issues, we propose a novel Semantic-Guided Triplet Co-training (SGTC) framework, which achieves high-end medical image segmentation by only annotating three orthogonal slices of a few volumetric samples, significantly alleviating the burden of radiologists. Our method consist of two main components. Specifically, to enable semantic-aware, fine-granular segmentation and enhance the quality of pseudo-labels, a novel semantic-guided auxiliary learning mechanism is proposed based on the pretrained CLIP. In addition, focusing on a more challenging but clinically realistic scenario, a new triple-view disparity training strategy is proposed, which uses sparse annotations (i.e., only three labeled slices of a few volumes) to perform co-training between three sub-networks, significantly improving the robustness. Extensive experiments on three public medical datasets demonstrate that our method outperforms most state-of-the-art semi-supervised counterparts under sparse annotation settings. The source code is available at https://github.com/xmeimeimei/SGTC.

Linglan Zhao, Xuerui Zhang, Ke Yan et al.

Continual learning aims to incrementally acquire new concepts in data streams while resisting forgetting previous knowledge. With the rise of powerful pre-trained models (PTMs), there is a growing interest in training incremental learning systems using these foundation models, rather than learning from scratch. Existing works often view PTMs as a strong initial point and directly apply parameter-efficient tuning (PET) in the first session for adapting to downstream tasks. In the following sessions, most methods freeze model parameters for tackling forgetting issues. However, applying PET directly to downstream data cannot fully explore the inherent knowledge in PTMs. Additionally, freezing the parameters in incremental sessions hinders models' plasticity to novel concepts not covered in the first session. To solve the above issues, we propose a Slow And Fast parameter-Efficient tuning (SAFE) framework. In particular, to inherit general knowledge from foundation models, we include a transfer loss function by measuring the correlation between the PTM and the PET-applied model. After calibrating in the first session, the slow efficient tuning parameters can capture more informative features, improving generalization to incoming classes. Moreover, to further incorporate novel concepts, we strike a balance between stability and plasticity by fixing slow efficient tuning parameters and continuously updating the fast ones. Specifically, a cross-classification loss with feature alignment is proposed to circumvent catastrophic forgetting. During inference, we introduce an entropy-based aggregation strategy to dynamically utilize the complementarity in the slow and fast learners. Extensive experiments on seven benchmark datasets verify the effectiveness of our method by significantly surpassing the state-of-the-art.

Xiaoyu Bai, Fan Bai, Xiaofei Huo et al.

Identifying specific anatomical structures (\textit{e.g.}, lesions or landmarks) in medical images plays a fundamental role in medical image analysis. Exemplar-based landmark detection methods are receiving increasing attention since they can detect arbitrary anatomical points in inference while do not need landmark annotations in training. They use self-supervised learning to acquire a discriminative embedding for each voxel within the image. These approaches can identify corresponding landmarks through nearest neighbor matching and has demonstrated promising results across various tasks. However, current methods still face challenges in: (1) differentiating voxels with similar appearance but different semantic meanings (\textit{e.g.}, two adjacent structures without clear borders); (2) matching voxels with similar semantics but markedly different appearance (\textit{e.g.}, the same vessel before and after contrast injection); and (3) cross-modality matching (\textit{e.g.}, CT-MRI landmark-based registration). To overcome these challenges, we propose universal anatomical embedding (UAE), which is a unified framework designed to learn appearance, semantic, and cross-modality anatomical embeddings. Specifically, UAE incorporates three key innovations: (1) semantic embedding learning with prototypical contrastive loss; (2) a fixed-point-based matching strategy; and (3) an iterative approach for cross-modality embedding learning. We thoroughly evaluated UAE across intra- and inter-modality tasks, including one-shot landmark detection, lesion tracking on longitudinal CT scans, and CT-MRI affine/rigid registration with varying field of view. Our results suggest that UAE outperforms state-of-the-art methods, offering a robust and versatile approach for landmark based medical image analysis tasks. Code and trained models are available at: \href{https://shorturl.at/bgsB3}

Yan Ke, Minqing Zhang, Xinpeng Zhang et al.

Reversible data hiding in encrypted domain (RDH-ED) schemes based on symmetric or public key encryption are mainly applied to the security of end-to-end communication. Aimed at providing reliable technical supports for multi-party security scenarios, a separable RDH-ED scheme for secret image sharing based on Chinese remainder theorem (CRT) is presented. In the application of (t, n) secret image sharing, the image is first shared into n different shares of ciphertext. Only when not less than t shares obtained, can the original image be reconstructed. In our scheme, additional data could be embedded into the image shares. To realize data extraction from the image shares and the reconstructed image separably, two data hiding methods are proposed: one is homomorphic difference expansion in encrypted domain (HDE-ED) that supports data extraction from the reconstructed image by utilizing the addition homomorphism of CRT secret sharing; the other is difference expansion in image shares (DE-IS) that supports the data extraction from the marked shares before image reconstruction. Experimental results demonstrate that the proposed scheme could not only maintain the security and the threshold function of secret sharing system, but also obtain a better reversibility and efficiency compared with most existing RDH-ED algorithms. The maximum embedding rate of HDE-ED could reach 0.5000 bits per pixel and the average embedding rate of DE-IS is 0.0545 bits per bit of ciphertext.

Jia Liu, Yan Ke, Yu Lei et al.

In the past few years, the Generative Adversarial Network (GAN) which proposed in 2014 has achieved great success. GAN has achieved many research results in the field of computer vision and natural language processing. Image steganography is dedicated to hiding secret messages in digital images, and has achieved the purpose of covert communication. Recently, research on image steganography has demonstrated great potential for using GAN and neural networks. In this paper we review different strategies for steganography such as cover modification, cover selection and cover synthesis by GANs, and discuss the characteristics of these methods as well as evaluation metrics and provide some possible future research directions in image steganography.

Yan Ke, Min-qing Zhang, Jia Liu et al.

This paper proposes a fully homomorphic encryption encapsulated difference expansion (FHEE-DE) scheme for reversible data hiding in encrypted domain (RDH-ED). In the proposed scheme, we use key-switching and bootstrapping techniques to control the ciphertext extension and decryption failure. To realize the data extraction directly from the encrypted domain without the private key, a key-switching based least-significant-bit (KS-LSB) data hiding method has been designed. In application, the user first encrypts the plaintext and uploads ciphertext to the server. Then the server performs data hiding by FHEE-DE and KS-LSB to obtain the marked ciphertext. Additional data can be extracted directly from the marked ciphertext by the server without the private key. The user can decrypt the marked ciphertext to obtain the marked plaintext. Then additional data or plaintext can be obtained from the marked plaintext by using the standard DE extraction or recovery. A fidelity constraint of DE is introduced to reduce the distortion of the marked plaintext. FHEE-DE enables the server to implement FHEE-DE recovery or extraction on the marked ciphertext, which returns the ciphertext of original plaintext or additional data to the user. In addition, we simplified the homomorphic operations of the proposed universal FHEE-DE to obtain an efficient version. The Experimental results demonstrate that the embedding capacity, fidelity, and reversibility of the proposed scheme are superior to existing RDH-ED methods, and fully separability is achieved without reducing the security of encryption.

Yan Ke, Jia Liu, Min-qing Zhang et al.

This paper focuses on several theoretical issues and principles in steganography security, and defines four security levels by analyzing the corresponding algorithm instances. In the theoretical analysis, we discuss the differences between steganography security and watermarking security. The two necessary conditions for the steganography security are obtained. Under the current technology situation, we then analyze the indistinguishability of the cover and stego-cover, and consider that the steganography security should rely on the key secrecy with algorithms open. By specifying the role of key in steganography, the necessary conditions for a secure steganography algorithm in theory are formally presented. When analyzing the security instances, we have classified the steganalysis attacks according to their variable access to the steganography system, and then defined the four security levels. The higher level security one has, the higher level attacks one can resist. We have also presented algorithm instances based on current technical conditions, and analyzed their data hiding process, security level, and practice requirements.

Zhuo Zhang, Jia Liu, Yan Ke et al.

In this paper, a novel data-driven information hiding scheme called generative steganography by sampling (GSS) is proposed. Unlike in traditional modification-based steganography, in our method the stego image is directly sampled by a powerful generator: no explicit cover is used. Both parties share a secret key used for message embedding and extraction. The Jensen-Shannon divergence is introduced as a new criterion for evaluating the security of generative steganography. Based on these principles, we propose a simple practical generative steganography method that uses semantic image inpainting. The message is written in advance to an uncorrupted region that needs to be retained in the corrupted image. Then, the corrupted image with the secret message is fed into a Generator trained by a generative adversarial network (GAN) for semantic completion. Message loss and prior loss terms are proposed for penalizing message extraction error and unrealistic stego image. In our design, we first train a generator whose training target is the generation of new data samples from the same distribution as that of existing training data. Next, for the trained generator, backpropagation to the message and prior loss are introduced to optimize the coding of the input noise data for the generator. The presented experiments demonstrate the potential of the proposed framework based on both qualitative and quantitative evaluations of the generated stego images.

Jia Liu, Yan Ke, Yu Lei et al.

In order to keep the data secret, various techniques have been implemented to encrypt and decrypt the secret data. Cryptography is committed to the security of content, i.e. it cannot be restored with a given ciphertext. Steganography is to hiding the existence of a communication channel within a stego. However, it has been difficult to construct a cipher (cypher) that simultaneously satisfy both channel and content security for secure communication. Inspired by the Cardan grille, this paper presents a new generative framework for grille cipher. A digital cardan grille is used for message encryption and decryption. The ciphertext is directly sampled by a powerful generator without an explicit cover. Message loss and prior loss are proposed for penalizing message extraction error and unrealistic ciphertext. Jensen-Shannon Divergence is introduced as new criteria for channel security. A simple practical data-driven grille cipher is proposed using semantic image inpainting and generative adversarial network. Experimental results demonstrate the promising of the proposed method.

Jia Liu, Tanping Zhou, Zhuo Zhang et al.

In this paper, a new framework for construction of Cardan grille for information hiding is proposed. Based on the semantic image inpainting technique, the stego image are driven by secret messages directly. A mask called Digital Cardan Grille (DCG) for determining the hidden location is introduced to hide the message. The message is written to the corrupted region that needs to be filled in the corrupted image in advance. Then the corrupted image with secret message is feeded into a Generative Adversarial Network (GAN) for semantic completion. The adversarial game not only reconstruct the corrupted image , but also generate a stego image which contains the logic rationality of image content. The experimental results verify the feasibility of the proposed method.

Ming-ming Liu, Min-qing Zhang, Jia Liu et al.

Traditional image steganography modifies the content of the image more or less, it is hard to resist the detection of image steganalysis tools. To address this problem, a novel method named generative coverless information hiding method based on generative adversarial networks is proposed in this paper. The main idea of the method is that the class label of generative adversarial networks is replaced with the secret information as a driver to generate hidden image directly, and then extract the secret information from the hidden image through the discriminator. It's the first time that the coverless information hiding is achieved by generative adversarial networks. Compared with the traditional image steganography, this method does not modify the content of the original image. therefore, this method can resist image steganalysis tools effectively. In terms of steganographic capacity, anti-steganalysis, safety and reliability, the experimen shows that this hidden algorithm performs well.

Yan Ke, Minqing Zhang, Jia Liu et al.

The distortion in steganography that usually comes from the modification or recoding on the cover image during the embedding process leaves the steganalyzer with possibility of discriminating. Faced with such a risk, we propose generative steganography with Kerckhoffs' principle (GSK) in this letter. In GSK, the secret messages are generated by a cover image using a generator rather than embedded into the cover, thus resulting in no modifications in the cover. To ensure the security, the generators are trained to meet Kerckhoffs' principle based on generative adversarial networks (GAN). Everything about the GSK system, except the extraction key, is public knowledge for the receivers. The secret messages can be outputted by the generator if and only if the extraction key and the cover image are both inputted. In the generator training procedures, there are two GANs, Message- GAN and Cover-GAN, designed to work jointly making the generated results under the control of the extraction key and the cover image. We provide experimental results on the training process and give an example of the working process by adopting a generator trained on MNIST, which demonstrate that GSK can use a cover image without any modification to generate messages, and without the extraction key or the cover image, only meaningless results would be obtained.