Improvements to Gradient Descent Methods for Quantum Tensor Network Machine Learning
This work addresses practical training difficulties for researchers using quantum-inspired tensor networks in machine learning, though it appears incremental as it builds on existing techniques.
The paper tackled the problems of initialization and hyperparameter tuning in optimizing tensor networks for machine learning, introducing a 'copy node' method and gradient-based regularization that resulted in models with far fewer parameters and improved generalization performance.
Tensor networks have demonstrated significant value for machine learning in a myriad of different applications. However, optimizing tensor networks using standard gradient descent has proven to be difficult in practice. Tensor networks suffer from initialization problems resulting in exploding or vanishing gradients and require extensive hyperparameter tuning. Efforts to overcome these problems usually depend on specific network architectures, or ad hoc prescriptions. In this paper we address the problems of initialization and hyperparameter tuning, making it possible to train tensor networks using established machine learning techniques. We introduce a `copy node' method that successfully initializes arbitrary tensor networks, in addition to a gradient based regularization technique for bond dimensions. We present numerical results that show that the combination of techniques presented here produces quantum inspired tensor network models with far fewer parameters, while improving generalization performance.