Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation
This work addresses time-series compression for fault classification in industrial processes, offering a novel quaternion-based approach that shows incremental improvements over existing methods.
The authors tackled time-series compression by dividing data into segments, extracting statistical features, and encoding them as quaternions, then processing them with quaternion-valued neural networks trained using derived backpropagation rules. They applied this method to the Tennessee Eastman Dataset for fault classification, outperforming real-valued counterparts and baselines, and improved the classification benchmark from 81.43% to 83.90%.
We propose a novel quaternionic time-series compression methodology where we divide a long time-series into segments of data, extract the min, max, mean and standard deviation of these chunks as representative features and encapsulate them in a quaternion, yielding a quaternion valued time-series. This time-series is processed using quaternion valued neural network layers, where we aim to preserve the relation between these features through the usage of the Hamilton product. To train this quaternion neural network, we derive quaternion backpropagation employing the GHR calculus, which is required for a valid product and chain rule in quaternion space. Furthermore, we investigate the connection between the derived update rules and automatic differentiation. We apply our proposed compression method on the Tennessee Eastman Dataset, where we perform fault classification using the compressed data in two settings: a fully supervised one and in a semi supervised, contrastive learning setting. Both times, we were able to outperform real valued counterparts as well as two baseline models: one with the uncompressed time-series as the input and the other with a regular downsampling using the mean. Further, we could improve the classification benchmark set by SimCLR-TS from 81.43% to 83.90%.