Sketches for Time-Dependent Machine Learning
This work addresses the issue of model obsolescence in time-dependent data for machine learning practitioners, representing an incremental improvement.
The authors tackled the problem of machine learning models becoming obsolete due to changes in time series data by incorporating data distribution evolution into algorithms, resulting in significantly improved prediction capabilities in classification tasks without notable computational overhead.
Time series data can be subject to changes in the underlying process that generates them and, because of these changes, models built on old samples can become obsolete or perform poorly. In this work, we present a way to incorporate information about the current data distribution and its evolution across time into machine learning algorithms. Our solution is based on efficiently maintaining statistics, particularly the mean and the variance, of data features at different time resolutions. These data summarisations can be performed over the input attributes, in which case they can then be fed into the model as additional input features, or over latent representations learned by models, such as those of Recurrent Neural Networks. In classification tasks, the proposed techniques can significantly outperform the prediction capabilities of equivalent architectures with no feature / latent summarisations. Furthermore, these modifications do not introduce notable computational and memory overhead when properly adjusted.