Federated Learning with Only Positive Labels
This addresses a critical bottleneck in federated learning for scenarios with limited label access, such as privacy-sensitive applications, though it is an incremental improvement over existing methods.
The paper tackles the problem of multi-class classification in federated learning when each user only has positive data for one class, which can cause classifier collapse. The proposed Federated Averaging with Spreadout (FedAwS) framework uses geometric regularization to spread out class embeddings, achieving performance nearly matching that of conventional learning with negative labels.
We consider learning a multi-class classification model in the federated setting, where each user has access to the positive data associated with only a single class. As a result, during each federated learning round, the users need to locally update the classifier without having access to the features and the model parameters for the negative classes. Thus, naively employing conventional decentralized learning such as the distributed SGD or Federated Averaging may lead to trivial or extremely poor classifiers. In particular, for the embedding based classifiers, all the class embeddings might collapse to a single point. To address this problem, we propose a generic framework for training with only positive labels, namely Federated Averaging with Spreadout (FedAwS), where the server imposes a geometric regularizer after each round to encourage classes to be spreadout in the embedding space. We show, both theoretically and empirically, that FedAwS can almost match the performance of conventional learning where users have access to negative labels. We further extend the proposed method to the settings with large output spaces.