Methodology to Deploy CNN-Based Computer Vision Models on Immersive Wearable Devices
This enables real-time image processing on AR headsets for incorporating human input into AI models, but it is incremental as it adapts existing methods to a new platform.
The paper tackles the problem of deploying CNN models on AR headsets with limited processing power by training on computers and transferring optimized weights, achieving about 98% accuracy on MNIST with LeNet-5, similar to computer performance.
Convolutional Neural Network (CNN) models often lack the ability to incorporate human input, which can be addressed by Augmented Reality (AR) headsets. However, current AR headsets face limitations in processing power, which has prevented researchers from performing real-time, complex image recognition tasks using CNNs in AR headsets. This paper presents a method to deploy CNN models on AR headsets by training them on computers and transferring the optimized weight matrices to the headset. The approach transforms the image data and CNN layers into a one-dimensional format suitable for the AR platform. We demonstrate this method by training the LeNet-5 CNN model on the MNIST dataset using PyTorch and deploying it on a HoloLens AR headset. The results show that the model maintains an accuracy of approximately 98%, similar to its performance on a computer. This integration of CNN and AR enables real-time image processing on AR headsets, allowing for the incorporation of human input into AI models.