Catalog of Fault Types in Space Software from the ESVW investigation

Abstract

Space software may contain faults. However, as faults onboard a spacecraft might develop into extremely costly software and system failures during a mission, every effort is made to remove faults before costly failures commence. One building block of prevention methods is to reduce the risk of occurrence of runtime errors through verification. There are numerous tools on the market that promise to detect many of the various faults that can cause failures at runtime of the software. Yet it is unclear, what the different tools can achieve in practice, and what tools to best apply to real-world space software. Will they be able to detect all the faults they promise to detect, and will they differ in their fault detection capabilities? What efforts are typically associated with verification using such tools? Therefore, the product assurance department of the DLR Space Administration initiated the ESVW project (“Evaluierung von Softwareverifikationsmethoden und –Werkzeugen”) under contract number 50PS1502, executed by BSSE Software and System Engineering and etamax space GmbH. The aim of the ESVW project was to develop and evaluate a technique for investigating, comparing and characterizing different software verification methods and tools in the context of space flight software. As part of the evaluation activities, software faults and the problem reports generated by the different tools had to be harmonized. In particular, the way how tools report findings differ quite significantly. We needed a mapping from reported presumable faults to actual faults and false reports, and a classification scheme for known faults. This technical report presents the detailed work results regarding the fault ontology and mapping.