Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates
This addresses privacy and efficiency issues in federated learning for tabular data, though it is incremental as it builds on existing federated XGBoost methods.
The paper tackles the problem of training XGBoost in federated learning without sharing gradients to enhance privacy and communication efficiency, achieving performance comparable to state-of-the-art methods while reducing communication rounds and overhead by factors of 25x to 700x.
The privacy-sensitive nature of decentralized datasets and the robustness of eXtreme Gradient Boosting (XGBoost) on tabular data raise the needs to train XGBoost in the context of federated learning (FL). Existing works on federated XGBoost in the horizontal setting rely on the sharing of gradients, which induce per-node level communication frequency and serious privacy concerns. To alleviate these problems, we develop an innovative framework for horizontal federated XGBoost which does not depend on the sharing of gradients and simultaneously boosts privacy and communication efficiency by making the learning rates of the aggregated tree ensembles learnable. We conduct extensive evaluations on various classification and regression datasets, showing our approach achieves performance comparable to the state-of-the-art method and effectively improves communication efficiency by lowering both communication rounds and communication overhead by factors ranging from 25x to 700x. Project Page: https://flower.ai/blog/2023-04-19-xgboost-with-flower/