Tucker Decomposition Network: Expressive Power and Comparison
This work addresses the need for more expressive network architectures in machine learning and computer vision, though it is incremental as it builds on existing tensor decomposition methods.
The paper tackled the problem of limited expressive power in shallow neural networks by developing a deep network based on Tucker tensor decomposition, showing that it requires an exponential number of nodes in a shallow network to match its expressiveness, with experimental results demonstrating its usefulness in image classification.
Deep neural networks have achieved a great success in solving many machine learning and computer vision problems. The main contribution of this paper is to develop a deep network based on Tucker tensor decomposition, and analyze its expressive power. It is shown that the expressiveness of Tucker network is more powerful than that of shallow network. In general, it is required to use an exponential number of nodes in a shallow network in order to represent a Tucker network. Experimental results are also given to compare the performance of the proposed Tucker network with hierarchical tensor network and shallow network, and demonstrate the usefulness of Tucker network in image classification problems.