Semantic Face Compression for Metaverse: A Compact 3D Descriptor Based Approach
This work addresses efficient and flexible communication of semantic facial information for virtual avatars in the metaverse, enabling applications like digital human communication based on machine analysis.
The paper tackles the problem of compressing virtual avatar faces for metaverse communication by developing a semantic face compression scheme using compact 3D facial descriptors, achieving a significant improvement in rate-accuracy performance compared to the Versatile Video Coding standard.
In this letter, we envision a new metaverse communication paradigm for virtual avatar faces, and develop the semantic face compression with compact 3D facial descriptors. The fundamental principle is that the communication of virtual avatar faces primarily emphasizes the conveyance of semantic information. In light of this, the proposed scheme offers the advantages of being highly flexible, efficient and semantically meaningful. The semantic face compression, which allows the communication of the descriptors for artificial intelligence based understanding, could facilitate numerous applications without the involvement of humans in metaverse. The promise of the proposed paradigm is also demonstrated by performance comparisons with the state-of-the-art video coding standard, Versatile Video Coding. A significant improvement in terms of rate-accuracy performance has been achieved. The proposed scheme is expected to enable numerous applications, such as digital human communication based on machine analysis, and to form the cornerstone of interaction and communication in the metaverse.