Disentangle Perceptual Learning through Online Contrastive Learning
This work addresses a bottleneck in perceptual learning for image transformation tasks, offering an incremental improvement by refining feature selection from pre-trained networks.
The paper tackles the problem of suboptimal perceptual quality in image transformation tasks caused by irrelevant features in pre-trained classification networks, and proposes an online contrastive learning method to disentangle perception-relevant dimensions, resulting in superior performance over existing approaches in terms of human visual perception.
Pursuing realistic results according to human visual perception is the central concern in the image transformation tasks. Perceptual learning approaches like perceptual loss are empirically powerful for such tasks but they usually rely on the pre-trained classification network to provide features, which are not necessarily optimal in terms of visual perception of image transformation. In this paper, we argue that, among the features representation from the pre-trained classification network, only limited dimensions are related to human visual perception, while others are irrelevant, although both will affect the final image transformation results. Under such an assumption, we try to disentangle the perception-relevant dimensions from the representation through our proposed online contrastive learning. The resulted network includes the pre-training part and a feature selection layer, followed by the contrastive learning module, which utilizes the transformed results, target images, and task-oriented distorted images as the positive, negative, and anchor samples, respectively. The contrastive learning aims at activating the perception-relevant dimensions and suppressing the irrelevant ones by using the triplet loss, so that the original representation can be disentangled for better perceptual quality. Experiments on various image transformation tasks demonstrate the superiority of our framework, in terms of human visual perception, to the existing approaches using pre-trained networks and empirically designed losses.