Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
This provides a theoretical foundation for hyperparameter optimization in multi-task regression, though it is incremental as it extends existing random matrix theory to this specific context.
The authors developed a theoretical framework using random matrix theory to analyze multi-task regression performance under high-dimensional, non-Gaussian data, deriving closed-form solutions and error estimations that improved univariate models in time series forecasting.
In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information.