Epoch-evolving Gaussian Process Guided Learning
This work addresses optimization efficiency for deep learning practitioners, though it appears incremental as it builds on existing batch-based methods with a novel guidance scheme.
The paper tackles the problem of inefficient optimization in deep learning by proposing epoch-evolving Gaussian Process Guided Learning (GPGL), which encodes correlation between batch-level and global data distributions as context labels renewed each epoch; this approach outperforms existing batch-based state-of-the-art models on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets.
In this paper, we propose a novel learning scheme called epoch-evolving Gaussian Process Guided Learning (GPGL), which aims at characterizing the correlation information between the batch-level distribution and the global data distribution. Such correlation information is encoded as context labels and needs renewal every epoch. With the guidance of the context label and ground truth label, GPGL scheme provides a more efficient optimization through updating the model parameters with a triangle consistency loss. Furthermore, our GPGL scheme can be further generalized and naturally applied to the current deep models, outperforming the existing batch-based state-of-the-art models on mainstream datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) remarkably.