UniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Data
This addresses the problem of data heterogeneity in FL for real-world deployments, especially in resource-constrained settings, but it is incremental as it builds on existing regularization methods.
The paper tackles performance degradation in Federated Learning (FL) due to non-IID data by proposing UniVarFL, a framework that uses classifier variance and hyperspherical uniformity regularization to emulate IID-like training, resulting in improved accuracy on benchmark datasets.
Federated Learning (FL) often suffers from severe performance degradation when faced with non-IID data, largely due to local classifier bias. Traditional remedies such as global model regularization or layer freezing either incur high computational costs or struggle to adapt to feature shifts. In this work, we propose UniVarFL, a novel FL framework that emulates IID-like training dynamics directly at the client level, eliminating the need for global model dependency. UniVarFL leverages two complementary regularization strategies during local training: Classifier Variance Regularization, which aligns class-wise probability distributions with those expected under IID conditions, effectively mitigating local classifier bias; and Hyperspherical Uniformity Regularization, which encourages a uniform distribution of feature representations across the hypersphere, thereby enhancing the model's ability to generalize under diverse data distributions. Extensive experiments on multiple benchmark datasets demonstrate that UniVarFL outperforms existing methods in accuracy, highlighting its potential as a highly scalable and efficient solution for real-world FL deployments, especially in resource-constrained settings. Code: https://github.com/sunnyinAI/UniVarFL