Tetra-AML: Automatic Machine Learning via Tensor Networks
This addresses the computational and financial costs of deploying neural networks for commercial applications, though it appears incremental as it builds on existing tensor network and compression techniques.
The authors tackled the challenge of optimizing and deploying large neural networks by introducing Tetra-AML, an automated machine learning toolbox that uses tensor networks for architecture search, hyperparameter optimization, and model compression. They demonstrated superior performance compared to Bayesian optimization on CIFAR-10 and compressed ResNet-18 to use 14.5 times less memory with only a 3.2% accuracy loss.
Neural networks have revolutionized many aspects of society but in the era of huge models with billions of parameters, optimizing and deploying them for commercial applications can require significant computational and financial resources. To address these challenges, we introduce the Tetra-AML toolbox, which automates neural architecture search and hyperparameter optimization via a custom-developed black-box Tensor train Optimization algorithm, TetraOpt. The toolbox also provides model compression through quantization and pruning, augmented by compression using tensor networks. Here, we analyze a unified benchmark for optimizing neural networks in computer vision tasks and show the superior performance of our approach compared to Bayesian optimization on the CIFAR-10 dataset. We also demonstrate the compression of ResNet-18 neural networks, where we use 14.5 times less memory while losing just 3.2% of accuracy. The presented framework is generic, not limited by computer vision problems, supports hardware acceleration (such as with GPUs and TPUs) and can be further extended to quantum hardware and to hybrid quantum machine learning models.