Generative Hints
This work addresses the problem of improving invariance learning in vision models for researchers and practitioners, offering a novel but incremental enhancement over standard data augmentation.
The paper tackles the limitation of data augmentation in fully capturing invariances by proposing generative hints, a training methodology that enforces known invariances across the entire input space using virtual examples from a generative model, resulting in up to 1.78% top-1 accuracy improvement on fine-grained visual classification benchmarks and an average 1.286% boost on the CheXpert X-ray dataset.
Data augmentation is widely used in vision to introduce variation and mitigate overfitting, through enabling models to learn invariant properties, such as spatial invariance. However, these properties are not fully captured by data augmentation alone, since it attempts to learn the property on transformations of the training data only. We propose generative hints, a training methodology that directly enforces known invariances in the entire input space. Our approach leverages a generative model trained on the training set to approximate the input distribution and generate unlabeled images, which we refer to as virtual examples. These virtual examples are used to enforce functional properties known as hints. In generative hints, although the training dataset is fully labeled, the model is trained in a semi-supervised manner on both the classification and hint objectives, using the unlabeled virtual examples to guide the model in learning the desired hint. Across datasets, architectures, and loss functions, generative hints consistently outperform standard data augmentation when learning the same property. On popular fine-grained visual classification benchmarks, we achieved up to 1.78% top-1 accuracy improvement (0.63% on average) over fine-tuned models with data augmentation and an average performance boost of 1.286% on the CheXpert X-ray dataset.