Enhanced NIRMAL Optimizer With Damped Nesterov Acceleration: A Comparative Analysis
This is an incremental improvement for machine learning practitioners, offering a more stable optimizer for image classification tasks.
This paper tackles the problem of improving optimization for deep learning by introducing Enhanced NIRMAL, an optimizer with damped Nesterov acceleration, which achieved a test accuracy of 46.06% and the lowest test loss of 1.960435 on CIFAR-100, surpassing the original NIRMAL and closely rivaling SGD with Momentum.
This study introduces the Enhanced NIRMAL (Novel Integrated Robust Multi-Adaptation Learning with Damped Nesterov Acceleration) optimizer, an improved version of the original NIRMAL optimizer. By incorporating an $(α, r)$-damped Nesterov acceleration mechanism, Enhanced NIRMAL improves convergence stability while retaining chess-inspired strategies of gradient descent, momentum, stochastic perturbations, adaptive learning rates, and non-linear transformations. We evaluate Enhanced NIRMAL against Adam, SGD with Momentum, Nesterov, and the original NIRMAL on four benchmark image classification datasets: MNIST, FashionMNIST, CIFAR-10, and CIFAR-100, using tailored convolutional neural network (CNN) architectures. Enhanced NIRMAL achieves a test accuracy of 46.06\% and the lowest test loss (1.960435) on CIFAR-100, surpassing the original NIRMAL (44.34\% accuracy) and closely rivaling SGD with Momentum (46.43\% accuracy). These results underscore Enhanced NIRMAL's superior generalization and stability, particularly on complex datasets.