Parallel Stochastic Newton Method
Provides a parallel optimization method for large-scale machine learning problems, but the improvements are incremental over existing stochastic Newton methods.
The paper proposes a parallel stochastic Newton method (PSN) for minimizing smooth convex functions, showing conditions for acceleration over serial methods and demonstrating efficiency on empirical risk minimization problems.
We propose a parallel stochastic Newton method (PSN) for minimizing unconstrained smooth convex functions. We analyze the method in the strongly convex case, and give conditions under which acceleration can be expected when compared to its serial counterpart. We show how PSN can be applied to the empirical risk minimization problem, and demonstrate the practical efficiency of the method through numerical experiments and models of simple matrix classes.