Investigating the Effect of Intraclass Variability in Temporal Ensembling
This incremental study addresses the problem of optimizing Temporal Ensembling for researchers working with datasets having high intraclass variability.
The paper investigates how intraclass variability affects Temporal Ensembling, a semi-supervised method for training deep neural networks with limited labeled data, finding that high variability leads to a significant accuracy drop and that more seed images improve accuracy consistently.
Temporal Ensembling is a semi-supervised approach that allows training deep neural network models with a small number of labeled images. In this paper, we present our preliminary study on the effect of intraclass variability on temporal ensembling, with a focus on seed size and seed type, respectively. Through our experiments we find that (a) there is a significant drop in accuracy with datasets that offer high intraclass variability, (b) more seed images offer consistently higher accuracy across the datasets, and (c) seed type indeed has an impact on the overall efficiency, where it produces a spectrum of accuracy both lower and higher. Additionally, based on our experiments, we also find KMNIST to be a competitive baseline for temporal ensembling.