Decentralized Dynamic Discriminative Dictionary Learning
This addresses distributed learning for networks, but it appears incremental as it adapts an existing algorithm to a specific setting.
The paper tackles the problem of discriminative dictionary learning in a distributed online setting, where a network of agents learns a common dictionary and model parameters from sequential observations, and shows that decisions from their algorithm asymptotically achieve a first-order stationarity condition on average.
We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn a common set of dictionary elements of a feature space and model parameters while sequentially receiving observations. We formulate this problem as a distributed stochastic program with a non-convex objective and present a block variant of the Arrow-Hurwicz saddle point algorithm to solve it. Using Lagrange multipliers to penalize the discrepancy between them, only neighboring nodes exchange model information. We show that decisions made with this saddle point algorithm asymptotically achieve a first-order stationarity condition on average.