TTOpt: A Maximum Volume Quantized Tensor Train-based Optimization and its Application to Reinforcement Learning
This work addresses optimization challenges in machine learning, particularly for high-dimensional problems, though it appears incremental as it builds on existing tensor train and optimization techniques.
The authors tackled the problem of optimizing multidimensional functions and reinforcement learning tasks by introducing TTOpt, a method combining quantized tensor train representation with a maximum matrix volume principle, which outperformed evolutionary-based methods in function evaluations and execution time, often by significant margins.
We present a novel procedure for optimization based on the combination of efficient quantized tensor train representation and a generalized maximum matrix volume principle. We demonstrate the applicability of the new Tensor Train Optimizer (TTOpt) method for various tasks, ranging from minimization of multidimensional functions to reinforcement learning. Our algorithm compares favorably to popular evolutionary-based methods and outperforms them by the number of function evaluations or execution time, often by a significant margin.