Federated Variational Inference Methods for Structured Latent Variable Models
This work addresses the problem of enabling federated learning for structured latent variable models, which is incremental as it adapts existing variational inference methods to a distributed context.
The paper tackled the limitation of existing federated learning methods in handling structured probabilistic models by proposing a solution based on structured variational inference adapted for federated settings, including a communication-efficient variant, and demonstrated effectiveness through comparisons with hierarchical Bayesian neural networks and topic models.
Federated learning methods enable model training across distributed data sources without data leaving their original locations and have gained increasing interest in various fields. However, existing approaches are limited, excluding many structured probabilistic models. We present a general and elegant solution based on structured variational inference, widely used in Bayesian machine learning, adapted for the federated setting. Additionally, we provide a communication-efficient variant analogous to the canonical FedAvg algorithm. The proposed algorithms' effectiveness is demonstrated, and their performance is compared with hierarchical Bayesian neural networks and topic models.