Fuzzy hyperparameters update in a second order optimization
This work addresses optimization efficiency for machine learning practitioners, but it appears incremental as it builds on existing second-order methods with fuzzy logic enhancements.
The paper tackles the problem of accelerating convergence in second-order optimization by introducing a hybrid approach that combines an online finite difference approximation of the diagonal Hessian matrix with fuzzy inferencing of hyperparameters, achieving competitive results.
This research will present a hybrid approach to accelerate convergence in a second order optimization. An online finite difference approximation of the diagonal Hessian matrix will be introduced, along with fuzzy inferencing of several hyperparameters. Competitive results have been achieved