ZetA: A Riemann Zeta-Scaled Extension of Adam for Deep Learning
This work addresses optimization challenges for deep learning practitioners, offering a computationally efficient alternative to Adam, though it appears incremental as it builds on existing methods like Adam and SAM.
The paper tackles the problem of improving generalization and robustness in deep learning by introducing ZetA, a novel optimizer that extends Adam with Riemann zeta function-based scaling, resulting in consistent test accuracy improvements over Adam on datasets like SVHN, CIFAR10, and noisy CIFAR10.
This work introduces ZetA, a novel deep learning optimizer that extends Adam by incorporating dynamic scaling based on the Riemann zeta function. To the best of our knowledge, ZetA is the first optimizer to apply zeta-based gradient scaling within deep learning optimization. The method improves generalization and robustness through a hybrid update mechanism that integrates adaptive damping, cosine similarity-based momentum boosting, entropy-regularized loss, and Sharpness-Aware Minimization (SAM)-style perturbations. Empirical evaluations on SVHN, CIFAR10, CIFAR100, STL10, and noisy CIFAR10 consistently show test accuracy improvements over Adam. All experiments employ a lightweight fully connected network trained for five epochs under mixed-precision settings. The results demonstrate that ZetA is a computationally efficient and robust alternative to Adam, particularly effective in noisy or high-granularity classification tasks.