Distributed Stochastic Multi-Task Learning with Graph Regularization
This work addresses the challenge of efficient distributed learning for related tasks in machine learning systems, presenting an incremental improvement over existing consensus-based approaches.
The paper tackles the problem of distributed multi-task learning across machines by proposing methods that use weighted averaging of messages to learn related but distinct tasks, avoiding consensus to a single task.
We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines. Uniform averaging or diminishing stepsize in these methods would yield consensus (single task) learning. We show how simply skewing the averaging weights or controlling the stepsize allows learning different, but related, tasks on the different machines.