Communication-Efficient and Interoperable Distributed Learning
This work addresses interoperability and privacy issues in distributed learning for heterogeneous systems, representing an incremental improvement over existing methods like FL and FSL.
The paper tackles the challenge of collaborative learning across heterogeneous model architectures by proposing a communication-efficient distributed learning framework that supports model heterogeneity and modular composition during inference, achieving superior communication efficiency compared to federated learning and federated split learning baselines while ensuring stable training performance.
Collaborative learning across heterogeneous model architectures presents significant challenges in ensuring interoperability and preserving privacy. We propose a communication-efficient distributed learning framework that supports model heterogeneity and enables modular composition during inference. To facilitate interoperability, all clients adopt a common fusion-layer output dimension, which permits each model to be partitioned into a personalized base block and a generalized modular block. Clients share their fusion-layer outputs, keeping model parameters and architectures private. Experimental results demonstrate that the framework achieves superior communication efficiency compared to federated learning (FL) and federated split learning (FSL) baselines, while ensuring stable training performance across heterogeneous architectures.