Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
This work addresses scalability and communication bottlenecks in VFL for distributed data applications, presenting an incremental improvement over standard formulations.
The paper tackles the problem of efficiently training models in Vertical Federated Learning (VFL) by reformulating it as a saddle point problem using Lagrangian methods, and demonstrates convergence with stochastic adaptations like compression and partial participation.
The objective of Vertical Federated Learning (VFL) is to collectively train a model using features available on different devices while sharing the same users. This paper focuses on the saddle point reformulation of the VFL problem via the classical Lagrangian function. We first demonstrate how this formulation can be solved using deterministic methods. More importantly, we explore various stochastic modifications to adapt to practical scenarios, such as employing compression techniques for efficient information transmission, enabling partial participation for asynchronous communication, and utilizing coordinate selection for faster local computation. We show that the saddle point reformulation plays a key role and opens up possibilities to use mentioned extension that seem to be impossible in the standard minimization formulation. Convergence estimates are provided for each algorithm, demonstrating their effectiveness in addressing the VFL problem. Additionally, alternative reformulations are investigated, and numerical experiments are conducted to validate performance and effectiveness of the proposed approach.