AdaFamily: A family of Adam-like adaptive gradient methods
This work addresses optimization challenges in deep learning for researchers and practitioners, but it appears incremental as it blends existing methods.
The authors tackled the problem of training deep neural networks by proposing AdaFamily, a family of adaptive gradient methods, and demonstrated that it outperforms Adam, AdaBelief, and AdaMomentum on standard image classification datasets.
We propose AdaFamily, a novel method for training deep neural networks. It is a family of adaptive gradient methods and can be interpreted as sort of a blend of the optimization algorithms Adam, AdaBelief and AdaMomentum. We perform experiments on standard datasets for image classification, demonstrating that our proposed method outperforms these algorithms.