Representation Learning of Multivariate Time Series using Attention and Adversarial Training
This work addresses the need for robust data generation in trustworthy machine learning, particularly for imbalanced datasets or counterfactual explanations, but it is incremental as it adapts existing GAN and Transformer methods to a less-explored domain.
The paper tackled the problem of generating realistic multivariate time series data by proposing a Transformer-based autoencoder with adversarial training, and the results showed that the generated signals had higher similarity to an exemplary dataset compared to a convolutional network approach.
A critical factor in trustworthy machine learning is to develop robust representations of the training data. Only under this guarantee methods are legitimate to artificially generate data, for example, to counteract imbalanced datasets or provide counterfactual explanations for blackbox decision-making systems. In recent years, Generative Adversarial Networks (GANs) have shown considerable results in forming stable representations and generating realistic data. While many applications focus on generating image data, less effort has been made in generating time series data, especially multivariate signals. In this work, a Transformer-based autoencoder is proposed that is regularized using an adversarial training scheme to generate artificial multivariate time series signals. The representation is evaluated using t-SNE visualizations, Dynamic Time Warping (DTW) and Entropy scores. Our results indicate that the generated signals exhibit higher similarity to an exemplary dataset than using a convolutional network approach.