Towards a universal neural network encoder for time series
This work addresses the challenge of processing scarcely- or non-labeled time series across different domains, offering a potentially useful tool for time series analysis, though it appears incremental as it builds on existing encoder and attention mechanisms.
The paper tackles the problem of creating a universal neural network encoder for time series that can generalize to unseen data types, using a convolutional neural network with a convolutional attention mechanism to produce fixed-length representations from variable-length inputs. Results show that the approach is competitive with state-of-the-art methods on a time series classification benchmark, often outperforming conceptually-matching approaches, with benefits in adaptation ease and efficiency for scarcely-labeled data.
We study the use of a time series encoder to learn representations that are useful on data set types with which it has not been trained on. The encoder is formed of a convolutional neural network whose temporal output is summarized by a convolutional attention mechanism. This way, we obtain a compact, fixed-length representation from longer, variable-length time series. We evaluate the performance of the proposed approach on a well-known time series classification benchmark, considering full adaptation, partial adaptation, and no adaptation of the encoder to the new data type. Results show that such strategies are competitive with the state-of-the-art, often outperforming conceptually-matching approaches. Besides accuracy scores, the facility of adaptation and the efficiency of pre-trained encoders make them an appealing option for the processing of scarcely- or non-labeled time series.