SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction
This addresses a specific bottleneck in multi-task learning for spatiotemporal prediction, offering an incremental improvement for researchers in that domain.
The paper tackles the problem of multi-task learning for spatiotemporal data prediction when task numbers are small and models are nonlinear, proposing a two-step suboptimal unitary method (SUM) that improves generalization on distant tasks and enables coKriging. Experiments show it outperforms low-rank tensor learning on small tasks and adapts well to coKriging.
The typical multi-task learning methods for spatio-temporal data prediction involve low-rank tensor computation. However, such a method have relatively weak performance when the task number is small, and we cannot integrate it into non-linear models. In this paper, we propose a two-step suboptimal unitary method (SUM) to combine a meta-learning strategy into multi-task models. In the first step, it searches for a global pattern by optimising the general parameters with gradient descents under constraints, which is a geological regularizer to enable model learning with less training data. In the second step, we derive an optimised model on each specific task from the global pattern with only a few local training data. Compared with traditional multi-task learning methods, SUM shows advantages of generalisation ability on distant tasks. It can be applied on any multi-task models with the gradient descent as its optimiser regardless if the prediction function is linear or not. Moreover, we can harness the model to enable traditional prediction model to make coKriging. The experiments on public datasets have suggested that our framework, when combined with current multi-task models, has a conspicuously better prediction result when the task number is small compared to low-rank tensor learning, and our model has a quite satisfying outcome when adjusting the current prediction models for coKriging.