Coordinate-Descent Diffusion Learning by Networked Agents
Provides theoretical insights for reducing computational complexity in distributed learning over networks, relevant for power-intensive large data applications.
This paper analyzes the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme, where each agent updates only a random subset of coordinates. The results show that steady-state performance is not always degraded, though convergence rate suffers some degradation.
This work examines the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme. In this setting, the adaptation step by each agent is limited to a random subset of the coordinates of its stochastic gradient vector. The selection of coordinates varies randomly from iteration to iteration and from agent to agent across the network. Such schemes are useful in reducing computational complexity at each iteration in power-intensive large data applications. They are also useful in modeling situations where some partial gradient information may be missing at random. Interestingly, the results show that the steady-state performance of the learning strategy is not always degraded, while the convergence rate suffers some degradation. The results provide yet another indication of the resilience and robustness of adaptive distributed strategies.