Challenges in Software Safety for Army Test and Evaluation

As the capabilities of software intensive systems grow so does the complexity of functions controlled via software. Similarly, software test and evaluation (T&E) efforts have become increasingly difficult to quantify and scope appropriately. T&E efforts for traditional programs have hinged on system level testing in realistic or simulated environments to verify and validate the systems. Application of these traditional methods to software intensive systems continues to hold value, but it no longer provides exhaustive data. A number of T&E deficiencies are surfacing in test programs for these software intensive systems as controllability and visibility related to software functionality decreases. The result is testing that can fail to uncover critical problems, potentially with catastrophic results. This article describes successes and shortcomings with current test and analysis methodologies for software intensive systems. As a part of the Army Test and Evaluation Command (ATEC), the author looks from an unbiased viewpoint at relevant current practices and the outlook for future T&E in regards to software safety. Recent examples of Army software test and analysis efforts, current Army T&E guidance for software safety, and a path forward for increasing confidence in software safety will be discussed. Key words: Residual risk; risk; software intensive systems; software safety; T&E current practices; test and analysis methodologies