Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products
This work addresses performance limitations in neural networks for tasks like image denoising and audio separation, though it appears incremental as it builds on vector-valued learning with new bilinear products.
The paper tackles the problem of modeling associations among adjacent scalars in neural networks by proposing a vector-valued architecture using arbitrary bilinear products, resulting in substantial improvements in multispectral image denoising and singing voice separation compared to conventional methods.
Vector-valued neural learning has emerged as a promising direction in deep learning recently. Traditionally, training data for neural networks (NNs) are formulated as a vector of scalars; however, its performance may not be optimal since associations among adjacent scalars are not modeled. In this paper, we propose a new vector neural architecture called the Arbitrary BIlinear Product Neural Network (ABIPNN), which processes information as vectors in each neuron, and the feedforward projections are defined using arbitrary bilinear products. Such bilinear products can include circular convolution, seven-dimensional vector product, skew circular convolution, reversed- time circular convolution, or other new products not seen in previous work. As a proof-of-concept, we apply our proposed network to multispectral image denoising and singing voice sepa- ration. Experimental results show that ABIPNN gains substantial improvements when compared to conventional NNs, suggesting that associations are learned during training.