Ru Zhang

Yu Yue, Yufeng Yuan, Qiying Yu et al.

We present VAPO, Value-based Augmented Proximal Policy Optimization framework for reasoning models., a novel framework tailored for reasoning models within the value-based paradigm. Benchmarked the AIME 2024 dataset, VAPO, built on the Qwen 32B pre-trained model, attains a state-of-the-art score of $\mathbf{60.4}$. In direct comparison under identical experimental settings, VAPO outperforms the previously reported results of DeepSeek-R1-Zero-Qwen-32B and DAPO by more than 10 points. The training process of VAPO stands out for its stability and efficiency. It reaches state-of-the-art performance within a mere 5,000 steps. Moreover, across multiple independent runs, no training crashes occur, underscoring its reliability. This research delves into long chain-of-thought (long-CoT) reasoning using a value-based reinforcement learning framework. We pinpoint three key challenges that plague value-based methods: value model bias, the presence of heterogeneous sequence lengths, and the sparsity of reward signals. Through systematic design, VAPO offers an integrated solution that effectively alleviates these challenges, enabling enhanced performance in long-CoT reasoning tasks.

Ru Zhang, Sheng Zou, Jianyi Liu et al.

Traditional steganalysis methods generally include two steps: feature extraction and classification.A variety of steganalysis algorithms based on CNN (Convolutional Neural Network) have appeared in recent years. Among them, the convolutional layer of the CNN model is usually used to extract steganographic features, and the fully connected layer is used for classification. Because the effectiveness of feature extraction seriously influences the accuracy of classification, designers generally improve the accuracy of steganographic detection by improving the convolutional layer. For example, common optimizing methods in convolutional layer include the improvement of convolution kernel, activation functions, pooling functions, network structures, etc. However, due to the complexity and unexplainability of convolutional layers, it is difficult to quantitatively analyze and compare the effectiveness of feature extraction. Therefore, this paper proposes the variation coefficient to evaluate the feature learning ability of convolutional layers. We select four typical image steganalysis models based CNN in spatial domain, such as Ye-Net, Yedroudj-Net, Zhu-Net, and SR-Net as use cases, and verify the validity of the variation coefficient through experiments. Moreover, according to the variation coefficient , a features modification layer is used to optimize the features before the fully connected layer of the CNN model , and the experimental results show that the detection accuracy of the four algorithms were improved differently.

Ru Zhang, Shiqi Dong, Jianyi Liu

Nowadays, there are plenty of works introducing convolutional neural networks (CNNs) to the steganalysis and exceeding conventional steganalysis algorithms. These works have shown the improving potential of deep learning in information hiding domain. There are also several works based on deep learning to do image steganography, but these works still have problems in capacity, invisibility and security. In this paper, we propose a novel CNN architecture named as \isgan to conceal a secret gray image into a color cover image on the sender side and exactly extract the secret image out on the receiver side. There are three contributions in our work: (i) we improve the invisibility by hiding the secret image only in the Y channel of the cover image; (ii) We introduce the generative adversarial networks to strengthen the security by minimizing the divergence between the empirical probability distributions of stego images and natural images. (iii) In order to associate with the human visual system better, we construct a mixed loss function which is more appropriate for steganography to generate more realistic stego images and reveal out more better secret images. Experiment results show that ISGAN can achieve start-of-art performances on LFW, Pascal VOC2012 and ImageNet datasets.