Egocentric Human Segmentation for Mixed Reality
This work addresses the need for more realistic virtual avatars in mixed reality to enhance sense of presence, but it is incremental as it builds upon existing ThunderNet architecture.
The paper tackles the problem of segmenting human body parts from egocentric video by creating a semi-synthetic dataset of over 15,000 images and implementing a deep learning semantic segmentation algorithm based on ThunderNet that achieves real-time performance (16 ms for 720x720 images).
The objective of this work is to segment human body parts from egocentric video using semantic segmentation networks. Our contribution is two-fold: i) we create a semi-synthetic dataset composed of more than 15, 000 realistic images and associated pixel-wise labels of egocentric human body parts, such as arms or legs including different demographic factors; ii) building upon the ThunderNet architecture, we implement a deep learning semantic segmentation algorithm that is able to perform beyond real-time requirements (16 ms for 720 x 720 images). It is believed that this method will enhance sense of presence of Virtual Environments and will constitute a more realistic solution to the standard virtual avatars.