Federated Neuro-Symbolic Learning
This work addresses the problem of integrating neuro-symbolic learning into federated learning for scenarios with data privacy constraints, representing an incremental advancement by adapting existing methods to a new setting.
The paper tackles the challenge of applying neuro-symbolic learning in federated settings by proposing FedNSL, a framework that uses latent variables as a communication medium and addresses rule distribution heterogeneity with a KL divergence constraint, resulting in improvements of 17% in unbalanced average training accuracy and 29% in unseen average testing accuracy over baselines.
Neuro-symbolic learning (NSL) models complex symbolic rule patterns into latent variable distributions by neural networks, which reduces rule search space and generates unseen rules to improve downstream task performance. Centralized NSL learning involves directly acquiring data from downstream tasks, which is not feasible for federated learning (FL). To address this limitation, we shift the focus from such a one-to-one interactive neuro-symbolic paradigm to one-to-many Federated Neuro-Symbolic Learning framework (FedNSL) with latent variables as the FL communication medium. Built on the basis of our novel reformulation of the NSL theory, FedNSL is capable of identifying and addressing rule distribution heterogeneity through a simple and effective Kullback-Leibler (KL) divergence constraint on rule distribution applicable under the FL setting. It further theoretically adjusts variational expectation maximization (V-EM) to reduce the rule search space across domains. This is the first incorporation of distribution-coupled bilevel optimization into FL. Extensive experiments based on both synthetic and real-world data demonstrate significant advantages of FedNSL compared to five state-of-the-art methods. It outperforms the best baseline by 17% and 29% in terms of unbalanced average training accuracy and unseen average testing accuracy, respectively.