FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matching
This work addresses domain generalization for federated learning systems dealing with imbalanced data, offering a scalable solution that is incremental in combining existing techniques like sharpness minimization and gradient alignment.
The paper tackles the problem of domain generalization under long-tailed class distributions and conflicting optimization objectives by introducing FedTAIL, a federated framework that uses sharpness-guided gradient matching, achieving state-of-the-art performance on standard benchmarks with improved robustness to domain shifts and label imbalance.
Domain Generalization (DG) seeks to train models that perform reliably on unseen target domains without access to target data during training. While recent progress in smoothing the loss landscape has improved generalization, existing methods often falter under long-tailed class distributions and conflicting optimization objectives. We introduce FedTAIL, a federated domain generalization framework that explicitly addresses these challenges through sharpness-guided, gradient-aligned optimization. Our method incorporates a gradient coherence regularizer to mitigate conflicts between classification and adversarial objectives, leading to more stable convergence. To combat class imbalance, we perform class-wise sharpness minimization and propose a curvature-aware dynamic weighting scheme that adaptively emphasizes underrepresented tail classes. Furthermore, we enhance conditional distribution alignment by integrating sharpness-aware perturbations into entropy regularization, improving robustness under domain shift. FedTAIL unifies optimization harmonization, class-aware regularization, and conditional alignment into a scalable, federated-compatible framework. Extensive evaluations across standard domain generalization benchmarks demonstrate that FedTAIL achieves state-of-the-art performance, particularly in the presence of domain shifts and label imbalance, validating its effectiveness in both centralized and federated settings. Code: https://github.com/sunnyinAI/FedTail