LESS-VFL: Communication-Efficient Feature Selection for Vertical Federated Learning
This addresses communication bottlenecks in distributed systems with vertically partitioned data, offering a practical solution for collaborative model training with reduced overhead.
The paper tackles the problem of communication inefficiency in vertical federated learning by proposing LESS-VFL, a feature selection method that removes unimportant features to improve generalization, efficiency, and explainability, achieving high accuracy and spurious feature removal at a fraction of the communication cost of other approaches.
We propose LESS-VFL, a communication-efficient feature selection method for distributed systems with vertically partitioned data. We consider a system of a server and several parties with local datasets that share a sample ID space but have different feature sets. The parties wish to collaboratively train a model for a prediction task. As part of the training, the parties wish to remove unimportant features in the system to improve generalization, efficiency, and explainability. In LESS-VFL, after a short pre-training period, the server optimizes its part of the global model to determine the relevant outputs from party models. This information is shared with the parties to then allow local feature selection without communication. We analytically prove that LESS-VFL removes spurious features from model training. We provide extensive empirical evidence that LESS-VFL can achieve high accuracy and remove spurious features at a fraction of the communication cost of other feature selection approaches.