Stochastic Learning Equation using Monotone Increasing Resolution of Quantization
This work addresses optimization challenges in quantized learning systems, offering a method for global convergence without relying on local constraints like Hessian conditions, which is incremental in advancing stochastic optimization techniques.
The paper tackles the problem of global optimization in quantized learning by proposing a learning equation with monotone increasing quantization resolution and analyzing its convergence using stochastic methods, showing that it achieves global optimization for domains satisfying the Lipschitz condition.
In this paper, we propose a quantized learning equation with a monotone increasing resolution of quantization and stochastic analysis for the proposed algorithm. According to the white noise hypothesis for the quantization error with dense and uniform distribution, we can regard the quantization error as i.i.d.\ white noise. Based on this, we show that the learning equation with monotonically increasing quantization resolution converges weakly as the distribution viewpoint. The analysis of this paper shows that global optimization is possible for a domain that satisfies the Lipschitz condition instead of local convergence properties such as the Hessian constraint of the objective function.