Modified online Newton step based on element wise multiplication
This incremental improvement enables handling large multi-class datasets on common desktop machines using second-order methods.
The paper tackled the challenge of storing large second-order matrices in online Newton step methods for large datasets by proposing a modified approach that uses element-wise arithmetic to maintain matrix dimensions, resulting in faster computations while achieving a mistake rate comparable to existing methods.
The second order method as Newton Step is a suitable technique in Online Learning to guarantee regret bound. The large data is a challenge in Newton method to store second order matrices as hessian. In this paper, we have proposed an modified online Newton step that store first and second order matrices of dimension m (classes) by d (features). we have used element wise arithmetic operation to retain matrices size same. The modified second order matrix size results in faster computations. Also, the mistake rate is at par with respect to popular methods in literature. The experiments outcome indicate that proposed method could be helpful to handle large multi class datasets in common desktop machines using second order method as Newton step.