Regularized Diffusion Adaptation via Conjugate Smoothing
This work addresses distributed optimization for adaptation and learning problems with weaker assumptions, making it applicable to a broader class of scenarios.
The paper tackles the problem of distributed Pareto optimization of aggregate costs with non-differentiable regularizers across a network of agents, showing that smoothing via infimal convolution yields solutions arbitrarily close to the original non-smooth problem.
The purpose of this work is to develop and study a distributed strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution while the regularizers are assumed deterministic, but are not required to be differentiable or even continuous. The individual, regularized, cost functions are distributed across a strongly-connected network of agents and the Pareto optimal solution is sought by appealing to a multi-agent diffusion strategy. To this end, the regularizers are smoothed by means of infimal convolution and it is shown that the Pareto solution of the approximate, smooth problem can be made arbitrarily close to the solution of the original, non-smooth problem. Performance bounds are established under conditions that are weaker than assumed before in the literature, and hence applicable to a broader class of adaptation and learning problems.