ADAM Optimization with Adaptive Batch Selection
This work addresses inefficiencies in neural network training for practitioners, but it is incremental as it builds on prior bandit-based approaches.
The paper tackled the problem of inefficient convergence in Adam optimization by introducing AdamCB, which integrates combinatorial bandit sampling to adaptively select batches, resulting in faster convergence and improved performance over existing methods.
Adam is a widely used optimizer in neural network training due to its adaptive learning rate. However, because different data samples influence model updates to varying degrees, treating them equally can lead to inefficient convergence. To address this, a prior work proposed adapting the sampling distribution using a bandit framework to select samples adaptively. While promising, the bandit-based variant of Adam suffers from limited theoretical guarantees. In this paper, we introduce Adam with Combinatorial Bandit Sampling (AdamCB), which integrates combinatorial bandit techniques into Adam to resolve these issues. AdamCB is able to fully utilize feedback from multiple samples at once, enhancing both theoretical guarantees and practical performance. Our regret analysis shows that AdamCB achieves faster convergence than Adam-based methods including the previous bandit-based variant. Numerical experiments demonstrate that AdamCB consistently outperforms existing methods.