Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation
This addresses communication efficiency for clients in federated learning systems, particularly in multilingual settings, but is incremental as it builds on existing FL and meta-learning approaches.
The paper tackled communication constraints in federated multilingual machine translation by proposing MetaSend, a meta-learning-based adaptive parameter selection method, which improved translation quality under limited communication budgets, as demonstrated on two NMT datasets.
Federated learning (FL) is a promising distributed machine learning paradigm that enables multiple clients to collaboratively train a global model. In this paper, we focus on a practical federated multilingual learning setup where clients with their own language-specific data aim to collaboratively construct a high-quality neural machine translation (NMT) model. However, communication constraints in practical network systems present challenges for exchanging large-scale NMT engines between FL parties. We propose a meta-learning-based adaptive parameter selection methodology, MetaSend, that improves the communication efficiency of model transmissions from clients during FL-based multilingual NMT training. Our approach learns a dynamic threshold for filtering parameters prior to transmission without compromising the NMT model quality, based on the tensor deviations of clients between different FL rounds. Through experiments on two NMT datasets with different language distributions, we demonstrate that MetaSend obtains substantial improvements over baselines in translation quality in the presence of a limited communication budget.