Semantic Video Communication via Multi-Scale Convolution and Dynamic Routing for Next-Generation Networks
It addresses the need for bandwidth-efficient video transmission and semantic understanding in next-generation networks, particularly for resource-constrained IoT edge devices.
The paper proposes a generative AI framework for semantic video communication that uses multi-scale temporal convolution and capsule-based dynamic routing to achieve efficient transmission and robust semantic alignment. On ActivityNet Captions, it achieves 42.9% Recall@0.5 and 41.1% mean IoU with O(T) complexity suitable for edge devices.
The exponential growth of video traffic demands novel semantic communication paradigms that transmit meaning rather than raw bits. We present a generative AI-enabled framework for semantic video communication addressing two critical challenges: efficient hierarchical temporal modeling for bandwidth-constrained transmission and robust semantic alignment between video content and natural language queries at network edge devices. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities with O(T) complexity suitable for resource-constrained IoT deployments. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations, enabling flexible modeling of non-monotonic semantic alignments essential for goal-oriented communication. These components are unified through a multi-task learning objective optimizing temporal boundary regression, cross-modal alignment, and capsule diversity. Experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU while maintaining computational efficiency critical for edge deployment.