Investigations on the inference optimization techniques and their impact on multiple hardware platforms for Semantic Segmentation
This work addresses inference efficiency for semantic segmentation in autonomous vehicles, but it is incremental as it applies existing methods to new hardware comparisons.
The paper tackled the problem of reducing inference time for semantic segmentation in self-driving by exploring optimization techniques like TensorFlow and TensorRT, resulting in comparisons of inference time and energy consumption across hardware platforms, including an embedded Nvidia Jetson TX1.
In this work, the task of pixel-wise semantic segmentation in the context of self-driving with a goal to reduce the inference time is explored. Fully Convolutional Network (FCN-8s, FCN-16s, and FCN-32s) with a VGG16 encoder architecture and skip connections is trained and validated on the Cityscapes dataset. Numerical investigations are carried out for several inference optimization techniques built into TensorFlow and TensorRT to quantify their impact on the inference time and network size. Finally, the trained network is ported on to an embedded platform (Nvidia Jetson TX1) and the inference time, as well as the total energy consumed for inference across hardware platforms, are compared.