Matrix Neural Networks
This addresses inefficiencies in handling non-vectorial data like matrices for researchers and practitioners in machine learning, though it appears incremental as it builds on existing neural network frameworks.
The authors tackled the problem of neural networks requiring vectorized inputs, which can lose spatial information and increase computational cost, by proposing matrix neural networks (MatNet) that directly process matrices. They demonstrated comparable performance to state-of-the-art methods on MNIST classification and image super-resolution tasks with reduced complexity.
Traditional neural networks assume vectorial inputs as the network is arranged as layers of single line of computing units called neurons. This special structure requires the non-vectorial inputs such as matrices to be converted into vectors. This process can be problematic. Firstly, the spatial information among elements of the data may be lost during vectorisation. Secondly, the solution space becomes very large which demands very special treatments to the network parameters and high computational cost. To address these issues, we propose matrix neural networks (MatNet), which takes matrices directly as inputs. Each neuron senses summarised information through bilinear mapping from lower layer units in exactly the same way as the classic feed forward neural networks. Under this structure, back prorogation and gradient descent combination can be utilised to obtain network parameters efficiently. Furthermore, it can be conveniently extended for multimodal inputs. We apply MatNet to MNIST handwritten digits classification and image super resolution tasks to show its effectiveness. Without too much tweaking MatNet achieves comparable performance as the state-of-the-art methods in both tasks with considerably reduced complexity.