Living in a Material World: Learning Material Properties from Full-Waveform Flash Lidar Data for Semantic Segmentation
This work addresses the problem of improving scene understanding for robotics or autonomous systems by leveraging material properties from lidar data, but it is incremental as it builds on existing lidar and classification methods.
The paper investigates whether material types can be determined from full-waveform flash lidar data and finds that a temporal convolutional neural network classifier generally performs better across a wider range of materials, though factors like angle of incidence and material similarity can hinder performance.
Advances in lidar technology have made the collection of 3D point clouds fast and easy. While most lidar sensors return per-point intensity (or reflectance) values along with range measurements, flash lidar sensors are able to provide information about the shape of the return pulse. The shape of the return waveform is affected by many factors, including the distance that the light pulse travels and the angle of incidence with a surface. Importantly, the shape of the return waveform also depends on the material properties of the reflecting surface. In this paper, we investigate whether the material type or class can be determined from the full-waveform response. First, as a proof of concept, we demonstrate that the extra information about material class, if known accurately, can improve performance on scene understanding tasks such as semantic segmentation. Next, we learn two different full-waveform material classifiers: a random forest classifier and a temporal convolutional neural network (TCN) classifier. We find that, in some cases, material types can be distinguished, and that the TCN generally performs better across a wider range of materials. However, factors such as angle of incidence, material colour, and material similarity may hinder overall performance.