FedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availability
This addresses challenges in federated learning for applications with strict data privacy regulations, offering an incremental improvement to mitigate issues like gradient noise and client-drift.
The paper tackles the problem of client-drift and variance in federated learning due to heterogeneous data and limited client availability, proposing FedOAED which incorporates an on-device autoencoder denoiser and shows consistent outperformance over state-of-the-art baselines on multiple vision datasets under Non-IID settings.
Over the last few decades, machine learning (ML) and deep learning (DL) solutions have demonstrated their potential across many applications by leveraging large amounts of high-quality data. However, strict data-sharing regulations such as the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA) have prevented many data-driven applications from being realised. Federated Learning (FL), in which raw data never leaves local devices, has shown promise in overcoming these limitations. Although FL has grown rapidly in recent years, it still struggles with heterogeneity, which produces gradient noise, client-drift, and increased variance from partial client participation. In this paper, we propose FedOAED, a novel federated learning algorithm designed to mitigate client-drift arising from multiple local training updates and the variance induced by partial client participation. FedOAED incorporates an on-device autoencoder denoiser on the client side to mitigate client-drift and variance resulting from heterogeneous data under limited client availability. Experiments on multiple vision datasets under Non-IID settings demonstrate that FedOAED consistently outperforms state-of-the-art baselines.