Contrast with Major Classifier Vectors for Federated Medical Relation Extraction with Heterogeneous Label Distribution
This addresses data privacy and label imbalance issues in federated learning for medical applications, representing an incremental improvement over existing methods.
The paper tackles the problem of heterogeneous label distribution in federated medical relation extraction by introducing major classifier vectors to prevent local overfitting, achieving state-of-the-art performance on three datasets.
Federated medical relation extraction enables multiple clients to train a deep network collaboratively without sharing their raw medical data. In order to handle the heterogeneous label distribution across clients, most of the existing works only involve enforcing regularization between local and global models during optimization. In this paper, we fully utilize the models of all clients and propose a novel concept of \textit{major classifier vectors}, where a group of class vectors is obtained in an ensemble rather than the weighted average method on the server. The major classifier vectors are then distributed to all clients and the local training of each client is Contrasted with Major Classifier vectors (FedCMC), so the local model is not prone to overfitting to the local label distribution. FedCMC requires only a small amount of additional transfer of classifier parameters without any leakage of raw data, extracted representations, and label distributions. Our extensive experiments show that FedCMC outperforms the other state-of-the-art FL algorithms on three medical relation extraction datasets.