Applying Deep Learning for cockpit segmentation in the context of mixed reality
This work addresses cockpit segmentation for mixed reality in a specific simulator context, but the approach is incremental and domain-specific.
The paper applies U-net and DeepLabV3+ convolutional neural networks to segment cockpit images from an off-highway truck simulator for mixed reality applications, achieving around 90% accuracy and identifying the best model.
Computer vision is an area that has been growing continuously. With the advance of technologies with a first-person view, new development opportunities have emerged inside the area. Mixed reality promotes virtual environments with objects from the physical world shown in real time. For that, it's necessary to be concerned with the immersion of the user in this simulated environment, increasingly seeking to bring it closer to a possible desired reality. This paper proposes the development of image processing in order to perform the segmentation of images to identify what is foreground and background in order to facilitate the union of virtual and real images. Thus, the present work obtain real images of the user using the off-highway truck simulator CAT793F, through a camera, to be able to perform the segmentation of such images with artificial intelligence techniques.The convolutional neural network architectures "U-net" and "DeepLabV3+" are applied to perform image segmentation. As a result, metrics with around 90% accuracy were presented and and the best model was determined.