Query-based Knowledge Transfer for Heterogeneous Learning Environments
This addresses the problem of inadequate client-specific learning in decentralized environments for applications like clinical benchmarks, though it appears incremental as an enhancement to existing collaborative learning approaches.
The paper tackles the problem of decentralized collaborative learning under data heterogeneity and privacy constraints by proposing Query-based Knowledge Transfer (QKT), which enables tailored knowledge acquisition for clients without direct data exchange, resulting in average performance improvements of 20.91% in single-class and 14.32% in multi-class query settings over existing methods.
Decentralized collaborative learning under data heterogeneity and privacy constraints has rapidly advanced. However, existing solutions like federated learning, ensembles, and transfer learning, often fail to adequately serve the unique needs of clients, especially when local data representation is limited. To address this issue, we propose a novel framework called Query-based Knowledge Transfer (QKT) that enables tailored knowledge acquisition to fulfill specific client needs without direct data exchange. QKT employs a data-free masking strategy to facilitate communication-efficient query-focused knowledge transfer while refining task-specific parameters to mitigate knowledge interference and forgetting. Our experiments, conducted on both standard and clinical benchmarks, show that QKT significantly outperforms existing collaborative learning methods by an average of 20.91\% points in single-class query settings and an average of 14.32\% points in multi-class query scenarios. Further analysis and ablation studies reveal that QKT effectively balances the learning of new and existing knowledge, showing strong potential for its application in decentralized learning.