Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models
This work addresses the challenge of improving sample efficiency in prediction tasks for domains like healthcare or finance where additional time-series data is available during training but not at test time, though it is incremental as it builds on existing privileged learning frameworks.
The paper tackles the problem of learning prediction models more efficiently by using time-series privileged information available only during training, and proves that this approach is more sample-efficient under certain assumptions, showing on real-world datasets that it outperforms classical learning, especially with scarce data.
We study prediction of future outcomes with supervised models that use privileged information during learning. The privileged information comprises samples of time series observed between the baseline time of prediction and the future outcome; this information is only available at training time which differs from the traditional supervised learning. Our question is when using this privileged data leads to more sample-efficient learning of models that use only baseline data for predictions at test time. We give an algorithm for this setting and prove that when the time series are drawn from a non-stationary Gaussian-linear dynamical system of fixed horizon, learning with privileged information is more efficient than learning without it. On synthetic data, we test the limits of our algorithm and theory, both when our assumptions hold and when they are violated. On three diverse real-world datasets, we show that our approach is generally preferable to classical learning, particularly when data is scarce. Finally, we relate our estimator to a distillation approach both theoretically and empirically.