Prototype-Based Layered Federated Cross-Modal Hashing
This addresses the problem of distributed cross-modal hashing for clients with privacy concerns, offering a personalized federated learning approach that is incremental in improving performance over existing methods.
The paper tackles the challenge of applying federated learning to cross-modal hashing under privacy constraints and issues like statistical and model heterogeneity, proposing a prototype-based layered method that introduces prototypes for similarity learning and a hypernetwork for personalized updates, with experimental results showing it outperforms state-of-the-art methods on benchmark datasets.
Recently, deep cross-modal hashing has gained increasing attention. However, in many practical cases, data are distributed and cannot be collected due to privacy concerns, which greatly reduces the cross-modal hashing performance on each client. And due to the problems of statistical heterogeneity, model heterogeneity, and forcing each client to accept the same parameters, applying federated learning to cross-modal hash learning becomes very tricky. In this paper, we propose a novel method called prototype-based layered federated cross-modal hashing. Specifically, the prototype is introduced to learn the similarity between instances and classes on server, reducing the impact of statistical heterogeneity (non-IID) on different clients. And we monitor the distance between local and global prototypes to further improve the performance. To realize personalized federated learning, a hypernetwork is deployed on server to dynamically update different layers' weights of local model. Experimental results on benchmark datasets show that our method outperforms state-of-the-art methods.