Harnessing the Power of Federated Learning in Federated Contextual Bandits
This work addresses a gap in federated learning for sequential decision-making, offering a more unified approach that could enhance efficiency and applicability in distributed systems, though it appears incremental in bridging existing FL methods with FCB.
The paper tackles the disconnection between canonical federated learning (FL) and federated contextual bandits (FCB) by proposing FedIGW, a novel FCB design that leverages inverse gap weighting, resulting in better integration of FL innovations such as flexible protocols, modularized analyses, and seamless appendages like personalization and privacy.
Federated learning (FL) has demonstrated great potential in revolutionizing distributed machine learning, and tremendous efforts have been made to extend it beyond the original focus on supervised learning. Among many directions, federated contextual bandits (FCB), a pivotal integration of FL and sequential decision-making, has garnered significant attention in recent years. Despite substantial progress, existing FCB approaches have largely employed their tailored FL components, often deviating from the canonical FL framework. Consequently, even renowned algorithms like FedAvg remain under-utilized in FCB, let alone other FL advancements. Motivated by this disconnection, this work takes one step towards building a tighter relationship between the canonical FL study and the investigations on FCB. In particular, a novel FCB design, termed FedIGW, is proposed to leverage a regression-based CB algorithm, i.e., inverse gap weighting. Compared with existing FCB approaches, the proposed FedIGW design can better harness the entire spectrum of FL innovations, which is concretely reflected as (1) flexible incorporation of (both existing and forthcoming) FL protocols; (2) modularized plug-in of FL analyses in performance guarantees; (3) seamless integration of FL appendages (such as personalization, robustness, and privacy). We substantiate these claims through rigorous theoretical analyses and empirical evaluations.