Test Framework Quality Assurance: Augmenting Agile Processes with Safety Standards

Quality of embedded systems is often demonstrated by performed tests and guaranteed by the quality of the tools used to perform them. Test automation is important in agile development and test frameworks can be considered mission-critical. Thus, it is important to ensure the quality of tools used for quality assurance.This thesis explores how industries with agile processes can learn from safety-related development with plan-driven processes for increased test framework quality. Safety standards often rely on plan-driven processes, focused on discipline in long term prospects with substantial documentation and extensive upfront plans and designs. Agile approaches instead focus on quick adaptation, where software is evolved, undergoes continuous improvements and is delivered incrementally. A case study was performed as an industry collaboration. A literature study extracted approaches from articles and safety standards. Analysis and processing resulted in candidate solutions, principles and practices iteratively refined for general applicability and the industrial context. Insights on implications and perceived industrial value resulted from a focus group, with qualitative and quantitative data collected through moderated group discussions and complementary activities. Finally, this thesis proposes guidelines intended to be generally applicable, with a suggested augmented agile process of sequential ”mini V-models” inherently controlled by Definition of Dones. A case-specific set of proposed guidelines extends the suggestion while embracing insights from the focus group. Also identified was the importance of perceiving the framework as a tool-chain and not a single tool, where interaction sequences and intermediate results can be identified and utilized for analysis and applicable measures. Future work could refine the proposed guidelines with an industrial dynamic validation, and also extend the literature study and expand the focus group for diverse contexts and industrial perspectives