Estimation of optimal encoding ladders for tiled 360° VR video in adaptive streaming systems
This addresses cost-optimal and immersive streaming for VR video providers and clients, but it is incremental as it builds on existing encoding and streaming methods.
The paper tackles the problem of optimizing encoding ladders for tiled 360° VR video in adaptive streaming systems by introducing a content-aware method that estimates visual distortion and resource costs, then uses integer linear programming to form optimal ladders. Experimental results show it delivers better objective quality compared to recommended ladders from professional providers.
Given the significant industrial growth of demand for virtual reality (VR), 360° video streaming is one of the most important VR applications that require cost-optimal solutions to achieve widespread proliferation of VR technology. Because of its inherent variability of data-intensive content types and its tiled-based encoding and streaming, 360° video requires new encoding ladders in adaptive streaming systems to achieve cost-optimal and immersive streaming experiences. In this context, this paper targets both the provider's and client's perspectives and introduces a new content-aware encoding ladder estimation method for tiled 360° VR video in adaptive streaming systems. The proposed method first categories a given 360° video using its features of encoding complexity and estimates the visual distortion and resource cost of each bitrate level based on the proposed distortion and resource cost models. An optimal encoding ladder is then formed using the proposed integer linear programming (ILP) algorithm by considering practical constraints. Experimental results of the proposed method are compared with the recommended encoding ladders of professional streaming service providers. Evaluations show that the proposed encoding ladders deliver better results compared to the recommended encoding ladders in terms of objective quality for 360° video, providing optimal encoding ladders using a set of service provider's constraint parameters.