Lightweight Task-Oriented Semantic Communication Empowered by Large-Scale AI Models
This work addresses efficiency and reliability issues in task-oriented semantic communication for real-time scenarios, representing an incremental improvement over existing knowledge distillation methods.
The paper tackles the challenge of high computational demands in large-scale AI models for real-time semantic communication by proposing a fast distillation method with a pre-stored compression mechanism and channel adaptive module, resulting in improved task accuracy, reduced model size, lower computation latency, and decreased training data requirements compared to baselines.
Recent studies have focused on leveraging large-scale artificial intelligence (LAI) models to improve semantic representation and compression capabilities. However, the substantial computational demands of LAI models pose significant challenges for real-time communication scenarios. To address this, this paper proposes utilizing knowledge distillation (KD) techniques to extract and condense knowledge from LAI models, effectively reducing model complexity and computation latency. Nevertheless, the inherent complexity of LAI models leads to prolonged inference times during distillation, while their lack of channel awareness compromises the distillation performance. These limitations make standard KD methods unsuitable for task-oriented semantic communication scenarios. To address these issues, we propose a fast distillation method featuring a pre-stored compression mechanism that eliminates the need for repetitive inference, significantly improving efficiency. Furthermore, a channel adaptive module is incorporated to dynamically adjust the transmitted semantic information based on varying channel conditions, enhancing communication reliability and adaptability. In addition, an information bottleneck-based loss function is derived to guide the fast distillation process. Simulation results verify that the proposed scheme outperform baselines in term of task accuracy, model size, computation latency, and training data requirements.