An end-to-end CNN framework for polarimetric vision tasks based on polarization-parameter-constructing network
This work addresses a domain-specific problem for researchers in polarimetric vision by providing an incremental improvement in processing polarization data.
The authors tackled the problem of underutilizing polarization information in convolutional neural networks for vision tasks by proposing an end-to-end CNN framework with a polarization-parameter-constructing network, achieving much higher mean-average-precision in object detection compared to existing methods.
Pixel-wise operations between polarimetric images are important for processing polarization information. For the lack of such operations, the polarization information cannot be fully utilized in convolutional neural network(CNN). In this paper, a novel end-to-end CNN framework for polarization vision tasks is proposed, which enables the networks to take full advantage of polarimetric images. The framework consists of two sub-networks: a polarization-parameter-constructing network (PPCN) and a task network. PPCN implements pixel-wise operations between images in the CNN form with 1x1 convolution kernels. It takes raw polarimetric images as input, and outputs polarization-parametric images to task network so as to complete a vison task. By training together, the PPCN can learn to provide the most suitable polarization-parametric images for the task network and the dataset. Taking faster R-CNN as task network, the experimental results show that compared with existing methods, the proposed framework achieves much higher mean-average-precision (mAP) in object detection task