Secure AI-Driven Super-Resolution for Real-Time Mixed Reality Applications
This addresses bandwidth and latency issues for real-time mixed reality applications, but it is incremental as it builds on existing super-resolution and encryption techniques.
The paper tackles bandwidth and latency challenges in real-time AR/VR streaming by downsampling and partially encrypting point cloud content at the server, then upscaling it with an ML-based super-resolution model at the client, achieving nearly linear reductions in bandwidth/latency and effective reconstruction with minimal error.
Immersive formats such as 360° and 6DoF point cloud videos require high bandwidth and low latency, posing challenges for real-time AR/VR streaming. This work focuses on reducing bandwidth consumption and encryption/decryption delay, two key contributors to overall latency. We design a system that downsamples point cloud content at the origin server and applies partial encryption. At the client, the content is decrypted and upscaled using an ML-based super-resolution model. Our evaluation demonstrates a nearly linear reduction in bandwidth/latency, and encryption/decryption overhead with lower downsampling resolutions, while the super-resolution model effectively reconstructs the original full-resolution point clouds with minimal error and modest inference time.