Distribution Estimation to Automate Transformation Policies for Self-Supervision
This addresses the challenge of selecting effective transformations for self-supervised learning in computer vision, though it appears incremental as it builds on existing pretext task methods.
The paper tackles the problem of ineffective image transformations in self-supervised learning by proposing a GAN-based framework to automatically find transformations not present in the dataset, which improves representation learning as evaluated on several visual recognition datasets.
In recent visual self-supervision works, an imitated classification objective, called pretext task, is established by assigning labels to transformed or augmented input images. The goal of pretext can be predicting what transformations are applied to the image. However, it is observed that image transformations already present in the dataset might be less effective in learning such self-supervised representations. Building on this observation, we propose a framework based on generative adversarial network to automatically find the transformations which are not present in the input dataset and thus effective for the self-supervised learning. This automated policy allows to estimate the transformation distribution of a dataset and also construct its complementary distribution from which training pairs are sampled for the pretext task. We evaluated our framework using several visual recognition datasets to show the efficacy of our automated transformation policy.