Plug-and-Play Parameter-Efficient Tuning of Embeddings for Federated Recommendation
This work addresses a critical bottleneck in federated recommendation systems for cloud-edge collaboration, offering a plug-and-play solution to enhance communication efficiency, though it is incremental by building on existing PEFT techniques.
The paper tackles the communication inefficiency in Federated Recommendation (FR) caused by large item embeddings, proposing a Parameter-Efficient Fine-Tuning (PEFT) framework that reduces embedding parameters transmitted. The result is a significant reduction in communication overhead while improving accuracy, as demonstrated in experiments across various FR models and datasets.
With the rise of cloud-edge collaboration, recommendation services are increasingly trained in distributed environments. Federated Recommendation (FR) enables such multi-end collaborative training while preserving privacy by sharing model parameters instead of raw data. However, the large number of parameters, primarily due to the massive item embeddings, significantly hampers communication efficiency. While existing studies mainly focus on improving the efficiency of FR models, they largely overlook the issue of embedding parameter overhead. To address this gap, we propose a FR training framework with Parameter-Efficient Fine-Tuning (PEFT) based embedding designed to reduce the volume of embedding parameters that need to be transmitted. Our approach offers a lightweight, plugin-style solution that can be seamlessly integrated into existing FR methods. In addition to incorporating common PEFT techniques such as LoRA and Hash-based encoding, we explore the use of Residual Quantized Variational Autoencoders (RQ-VAE) as a novel PEFT strategy within our framework. Extensive experiments across various FR model backbones and datasets demonstrate that our framework significantly reduces communication overhead while improving accuracy. The source code is available at https://github.com/young1010/FedPEFT.