Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling
This work addresses privacy-preserving and personalized modeling in federated settings, offering an incremental improvement by combining existing techniques like averaging and SGD for Gaussian processes.
The paper tackles the problem of federated learning for Gaussian process regression by proposing FGPR, which uses averaging for model aggregation and SGD for local computations, resulting in a global model that excels in personalization and multi-fidelity modeling, with theoretical convergence guarantees and demonstrated effectiveness in various applications.
In this paper, we propose \texttt{FGPR}: a Federated Gaussian process ($\mathcal{GP}$) regression framework that uses an averaging strategy for model aggregation and stochastic gradient descent for local client computations. Notably, the resulting global model excels in personalization as \texttt{FGPR} jointly learns a global $\mathcal{GP}$ prior across all clients. The predictive posterior then is obtained by exploiting this prior and conditioning on local data which encodes personalized features from a specific client. Theoretically, we show that \texttt{FGPR} converges to a critical point of the full log-likelihood function, subject to statistical error. Through extensive case studies we show that \texttt{FGPR} excels in a wide range of applications and is a promising approach for privacy-preserving multi-fidelity data modeling.