A probabilistic incremental proximal gradient method
This work addresses uncertainty estimation in optimization for machine learning and data science, but it is incremental as it builds on existing incremental proximal gradient methods.
The authors tackled the problem of uncertainty propagation in incremental proximal gradient optimization by proposing a probabilistic interpretation, resulting in a framework that enables the use of Bayesian filters like Kalman filters for large-scale regularized optimization.
In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model the update rules of the incremental proximal gradient method and develop a systematic approach to propagate the uncertainty of the solution estimate over iterations. The PIPG algorithm takes the form of Bayesian filtering updates for a state-space model constructed by using the cost function. Our framework makes it possible to utilize well-known exact or approximate Bayesian filters, such as Kalman or extended Kalman filters, to solve large-scale regularized optimization problems.