Test Process Assessment of Industrial Control Systems via Safety Standards

As more systems are becoming embedded hardware-based, challenges regarding software safety and considerable consequences of their failure arise. Various safety standards assure certain safety aspects of systems, addressing areas including testing. The safety standards chosen for this thesis are ISO/IEC/IEEE 29119-2 & 3, IEC 61508-1 & 3, ISO 13849-1 & 2, and ISO/IEC/IEEE 12207:2017. This thesis tackles the problem of compliance with safety standards by utilising a lightweight assessment method, leading to recommendations for improving the test process of an industrial control system. A case study is performed on an automation company to achieve the objectives of this thesis. The method used for the qualitative data analysis results in recommendations regarding the compliance of the company's test process with selected safety standards. As the final step, the execution of a focus group research leads to the industrial evaluation of the recommendations and assessment results. The company's development process fully complies with 22% and fails to comply with 58% of the extracted requirements from the selected safety standards. Furthermore, the thesis results in recommendations for improving the test process of an industrial control system. As a result of performing the case study, a method for a lightweight assessment of the development process of industrial control systems is achieved. The generic method follows five steps, firstly tabulating the data to attain assessment criteria and items, used by the assessment step to get a compliance degree per requirement. The analysis step comes next to shed light on areas of strength and weakness, leading to recommendations. The final step evaluates and refines the recommendations according to the results of a focus group. Further development of the method used in this thesis can lead to a generic method for assessing development processes, concerning safety standards, using limited resources. The results of this generic method can lead to recommendations for test process improvements of control systems via safety standards