On Performance Guarantees for Federated Learning with Personalized Constraints
It addresses the problem of handling heterogeneous resource or model constraints in federated learning for distributed agents, representing an incremental advance.
The paper tackles federated learning with personalized constraints by proposing PC-FedAvg, a method that allows agents to maintain cross-estimates without sharing constraint information, achieving communication-complexity rates of O(ε^{-2}) for suboptimality and O(ε^{-1}) for infeasibility.
Federated learning (FL) has emerged as a communication-efficient algorithmic framework for distributed learning across multiple agents. While standard FL formulations capture unconstrained or globally constrained problems, many practical settings involve heterogeneous resource or model constraints, leading to optimization problems with agent-specific feasible sets. Here, we study a personalized constrained federated optimization problem in which each agent is associated with a convex local objective and a private constraint set. We propose PC-FedAvg, a method in which each agent maintains cross-estimates of the other agents' variables through a multi-block local decision vector. Each agent updates all blocks locally, penalizing infeasibility only in its own block. Moreover, the cross-estimate mechanism enables personalization without requiring consensus or sharing constraint information among agents. We establish communication-complexity rates of $\mathcal{O}(ε^{-2})$ for suboptimality and $\mathcal{O}(ε^{-1})$ for agent-wise infeasibility. Preliminary experiments on the MNIST and CIFAR-10 datasets validate our theoretical findings.