Integrating Pre-Trained Language Model with Physical Layer Communications
This work addresses the problem of robust and efficient communication for on-device AI systems, representing an incremental improvement by combining existing methods like VQ-VAE and transformers with physical layer integration.
The paper tackles the challenge of integrating on-device AI communication frameworks with wireless systems by introducing a framework that incorporates physical layer functions, achieving a 50% reduction in transmission size and improved generalization and noise robustness in simulations.
The burgeoning field of on-device AI communication, where devices exchange information directly through embedded foundation models, such as language models (LMs), requires robust, efficient, and generalizable communication frameworks. However, integrating these frameworks with existing wireless systems and effectively managing noise and bit errors pose significant challenges. In this work, we introduce a practical ondevice AI communication framework, integrated with physical layer (PHY) communication functions, demonstrated through its performance on a link-level simulator. Our framework incorporates end-to-end training with channel noise to enhance resilience, incorporates vector quantized variational autoencoders (VQ-VAE) for efficient and robust communication, and utilizes pre-trained encoder-decoder transformers for improved generalization capabilities. Simulations, across various communication scenarios, reveal that our framework achieves a 50% reduction in transmission size while demonstrating substantial generalization ability and noise robustness under standardized 3GPP channel models.