The OPS-SAT benchmark for detecting anomalies in satellite telemetry
This work addresses a critical gap for space operations by enabling reproducible and objective evaluation of anomaly detection techniques, though it is incremental as it focuses on dataset creation rather than novel algorithmic advances.
The authors tackled the lack of publicly available datasets with ground-truth annotations for detecting anomalies in satellite telemetry by introducing the OPSSAT-AD benchmark dataset from the OPS-SAT CubeSat mission, providing baseline results from 30 machine learning algorithms and a set of quality metrics for validation.
Detecting anomalous events in satellite telemetry is a critical task in space operations. This task, however, is extremely time-consuming, error-prone and human dependent, thus automated data-driven anomaly detection algorithms have been emerging at a steady pace. However, there are no publicly available datasets of real satellite telemetry accompanied with the ground-truth annotations that could be used to train and verify anomaly detection supervised models. In this article, we address this research gap and introduce the AI-ready benchmark dataset (OPSSAT-AD) containing the telemetry data acquired on board OPS-SAT -- a CubeSat mission which has been operated by the European Space Agency which has come to an end during the night of 22--23 May 2024 (CEST). The dataset is accompanied with the baseline results obtained using 30 supervised and unsupervised classic and deep machine learning algorithms for anomaly detection. They were trained and validated using the training-test dataset split introduced in this work, and we present a suggested set of quality metrics which should be always calculated to confront the new algorithms for anomaly detection while exploiting OPSSAT-AD. We believe that this work may become an important step toward building a fair, reproducible and objective validation procedure that can be used to quantify the capabilities of the emerging anomaly detection techniques in an unbiased and fully transparent way.