Numerical Analysis of Automated Anomaly Detection Algorithms for Satellite Telemetry

Kurzfassung

As technology evolves and the complexity of satellites and the amount of available telemetry increases, the manual inspection of thousands of parameters in detail per satellite becomes less and less manageable. While automated processes such as Out-Of-Limit (OOL) checks, which verify if a parameter exceeds an upper or lower threshold, exist, they come with the drawback of needing to be defined manually and often being very coarse to detect subtle changes in the telemetry. As this is a known problem, many space agencies are developing anomaly detection systems using machine learning methods. We found that the main difficulty in developing such an algorithm, as has been done for the Automated Telemetry Health Monitoring System (ATHMoS) at German Space Operations Center (GSOC), is minimizing the number of false positives while still detecting anomalies at a sufficiently high rate. Also, computational cost needs to be minimized since the detection algorithm needs to run at least once per day for all parameters. Considering these important constraints specific to automatic anomaly detection for satellite telemetry, we analyse several algorithms commonly used, namely the LOF and LoOP algorithms, as well as, in more detail, the novel algorithm developed at GSOC named Outlier Probability Via Intrinsic Dimension (OPVID) with regards to these constraints. To this extent, we will use both academic and custom benchmarks based on artificial data and historic satellite telemetry to highlight the difficulties as well as provide solutions for choosing the right algorithms and their parameters for the wanted results. In addition to the analysis of the different algorithms for these benchmarks with mostly predefined features used as the algorithm input, we also want to provide a compact analysis of different features unique to their use case for satellite telemetry as an input to the OPVID algorithm. The results can also be extrapolated for various other algorithms. In an operational use case, these features need to be generic enough to describe every available telemetry parameter and, at the same time, provide a context for the engineers as the automated system should complement the operations team. In the result, we will see that the selection of the features has a large effect on both the false positive and true positive rate and is one of the keys to designing an anomaly detection system for an operational use case.