Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification
This work addresses person identification problems in biometrics, offering incremental improvements through novel fusion techniques.
The paper tackles multimodal biometric identification by proposing a method that uses modality-dedicated CNNs with feature-level fusion, achieving significant performance improvements over unimodal systems on databases like CMU Multi-PIE, BioCop, and BIOMDATA.
In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the convolutional layers, the fusion can be performed on the outputs of the fully-connected layers of the modality-specific CNNs without any loss of performance and with significant reduction in the number of parameters. We show that, using multiple CNNs with multimodal fusion at the feature-level, we significantly outperform systems that use unimodal representation. We study weighted feature, bilinear, and compact bilinear feature-level fusion algorithms for multimodal biometric person identification. Finally, We propose generalized compact bilinear fusion algorithm to deploy both the weighted feature fusion and compact bilinear schemes. We provide the results for the proposed algorithms on three challenging databases: CMU Multi-PIE, BioCop, and BIOMDATA.