ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
This work addresses the challenge of personalized learning in federated settings for applications where labeled data is scarce, though it is incremental as it builds on existing hierarchical Bayesian frameworks.
The paper tackles the problem of statistical heterogeneity in federated learning by developing personalized unsupervised learning algorithms for tasks like dimensionality reduction and diffusion models, demonstrating effective sample amplification through collaboration despite data differences.
Statistical heterogeneity of clients' local data is an important characteristic in federated learning, motivating personalized algorithms tailored to the local data statistics. Though there has been a plethora of algorithms proposed for personalized supervised learning, discovering the structure of local data through personalized unsupervised learning is less explored. We initiate a systematic study of such personalized unsupervised learning by developing algorithms based on optimization criteria inspired by a hierarchical Bayesian statistical framework. We develop adaptive algorithms that discover the balance between using limited local data and collaborative information. We do this in the context of two unsupervised learning tasks: personalized dimensionality reduction and personalized diffusion models. We develop convergence analyses for our adaptive algorithms which illustrate the dependence on problem parameters (e.g., heterogeneity, local sample size). We also develop a theoretical framework for personalized diffusion models, which shows the benefits of collaboration even under heterogeneity. We finally evaluate our proposed algorithms using synthetic and real data, demonstrating the effective sample amplification for personalized tasks, induced through collaboration, despite data heterogeneity.