Glass Segmentation with RGB-Thermal Image Pairs
This addresses glass segmentation for computer vision applications, but it is incremental as it builds on existing multi-modal fusion techniques with a new dataset.
The paper tackles glass segmentation by using paired RGB and thermal images, where glass is transparent to visible light but opaque to thermal energy, making it more distinguishable. The proposed neural network combines RGB-thermal pairs with a multi-modal fusion module based on attention, integrating CNN and transformer, and achieves effectiveness in qualitative and quantitative evaluations on a new dataset of 5551 image pairs.
This paper proposes a new glass segmentation method utilizing paired RGB and thermal images. Due to the large difference between the transmission property of visible light and that of the thermal energy through the glass where most glass is transparent to the visible light but opaque to thermal energy, glass regions of a scene are made more distinguishable with a pair of RGB and thermal images than solely with an RGB image. To exploit such a unique property, we propose a neural network architecture that effectively combines an RGB-thermal image pair with a new multi-modal fusion module based on attention, and integrate CNN and transformer to extract local features and non-local dependencies, respectively. As well, we have collected a new dataset containing 5551 RGB-thermal image pairs with ground-truth segmentation annotations. The qualitative and quantitative evaluations demonstrate the effectiveness of the proposed approach on fusing RGB and thermal data for glass segmentation. Our code and data are available at https://github.com/Dong-Huo/RGB-T-Glass-Segmentation.