Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Prediction
This addresses privacy-preserving model training for click-through rate prediction, offering an incremental improvement over existing federated learning methods.
The paper tackled slow convergence and poor results in federated learning for click-through rate prediction due to client heterogeneity and server learning rate tuning, proposing an online meta-learning method that significantly outperformed state-of-the-art in convergence speed and final quality.
In Federated Learning (FL) of click-through rate (CTR) prediction, users' data is not shared for privacy protection. The learning is performed by training locally on client devices and communicating only model changes to the server. There are two main challenges: (i) the client heterogeneity, making FL algorithms that use the weighted averaging to aggregate model updates from the clients have slow progress and unsatisfactory learning results; and (ii) the difficulty of tuning the server learning rate with trial-and-error methodology due to the big computation time and resources needed for each experiment. To address these challenges, we propose a simple online meta-learning method to learn a strategy of aggregating the model updates, which adaptively weighs the importance of the clients based on their attributes and adjust the step sizes of the update. We perform extensive evaluations on public datasets. Our method significantly outperforms the state-of-the-art in both the speed of convergence and the quality of the final learning results.