Personalized Federated Learning via Dual-Prompt Optimization and Cross Fusion
This addresses the problem of personalized model adaptation for decentralized clients in federated learning, representing an incremental advancement over existing prompt-based methods.
The paper tackles the challenge of data and domain heterogeneity in federated learning by proposing a personalized framework using dual-prompt optimization and cross fusion, achieving consistent performance improvements over state-of-the-art methods across nine datasets.
Federated learning (FL) enables collaborative model training across decentralized clients without sharing local data, but is challenged by heterogeneity in data, computation, and communication. Pretrained vision-language models (VLMs), with their strong generalization and lightweight tuning via prompts, offer a promising solution. However, existing federated prompt-learning methods rely only on text prompts and overlook joint label-domain distribution shifts. In this paper, we propose a personalized FL framework based on dual-prompt learning and cross fusion, termed pFedDC. Specifically, each client maintains both global and local prompts across vision and language modalities: global prompts capture common knowledge shared across the federation, while local prompts encode client-specific semantics and domain characteristics. Meanwhile, a cross-fusion module is designed to adaptively integrate prompts from different levels, enabling the model to generate personalized representations aligned with each client's unique data distribution. Extensive experiments across nine datasets with various types of heterogeneity show that pFedDC consistently outperforms state-of-the-art methods.