SEG-JPEG: Simple Visual Semantic Communications for Remote Operation of Automated Vehicles over Unreliable Wireless Networks
For remote operation of automated vehicles over unreliable wireless networks, this work provides a practical method to reduce bandwidth requirements and latency, potentially enabling large-scale deployment on constrained public 4G/5G networks.
The paper proposes SEG-JPEG, a semantic communication method that encodes road user segmentations into color-coded highlights in low-resolution grayscale imagery, reducing data rate by 50% while maintaining visual clarity. It achieves median glass-to-glass latency below 200 ms even at network data rates below 500 kbit/s, demonstrated on a real automated delivery vehicle over variable 4G.
Remote Operation is touted as being key to the rapid deployment of automated vehicles. Streaming imagery to control connected vehicles remotely currently requires a reliable, high throughput network connection, which can be limited in real-world remote operation deployments relying on public network infrastructure. This paper investigates how the application of computer vision assisted semantic communication can be used to circumvent data loss and corruption associated with traditional image compression techniques. By encoding the segmentations of detected road users into colour coded highlights within low resolution greyscale imagery, the required data rate can be reduced by 50% compared with conventional techniques, while maintaining visual clarity. This enables a median glass-to-glass latency of below 200 ms even when the network data rate is below 500 kbit/s, while clearly outlining salient road users to enhance situational awareness of the remote operator. The approach is demonstrated in an area of variable 4G mobile connectivity using an automated last-mile delivery vehicle. Results indicate that large-scale deployment of remotely operated automated vehicles could be possible even on the often constrained public 4G/5G mobile network, providing the potential to expedite the nationwide roll-out of automated vehicles.