Virtual Avatar Stream: a cost-down approach to the Metaverse experience
This provides a more inclusive and accessible entry point to the Metaverse for users limited by cost, though it is incremental in leveraging existing web technologies.
The project tackled the high cost barrier to Metaverse access by developing a web-based platform that uses WebGL and MediaPipe for real-time 3D face meshes, enabling users to engage with avatars via browser, microphone, and webcam without specialized hardware.
The Metaverse through VR headsets is a rapidly growing concept, but the high cost of entry currently limits access for many users. This project aims to provide an accessible entry point to the immersive Metaverse experience by leveraging web technologies. The platform developed allows users to engage with rendered avatars using only a web browser, microphone, and webcam. By employing the WebGL and MediaPipe face tracking AI model from Google, the application generates real-time 3D face meshes for users. It uses a client-to-client streaming cluster to establish a connection, and clients negotiate SRTP protocol through WebRTC for direct data streaming. Additionally, the project addresses backend challenges through an architecture that is serverless, distributive, auto-scaling, highly resilient, and secure. The platform offers a scalable, hardware-free solution for users to experience a near-immersive Metaverse, with the potential for future integration with game server clusters. This project provides an important step toward a more inclusive Metaverse accessible to a wider audience.