A Model For Measuring Improvement Of Security In Continuous Integration pipelines : Metrics and Four-Axis Maturity Driven DevSecOps (MFAM)

This Thesis researches the effect of adding security tools into CI pipelines. The thesis is based on "security by design" within the software development cycle. The CI pipeline is based on a relatively new topic area DevSecOps. In the CI pipeline, there are several ways in which security can be improved, as well as research to back this up. However, there are not many types of research on the measurement of improvement within this field. This thesis first investigates the literature on the topic of DevSecOps and Software Security. Following this is a systematic review of existing systems. It is then concluded that Dynamic application security testing (DAST) and Static application security testing (SAST) tools are added to the CI pipeline to further improve security by design. These improvements and additions are then measured with The Metrics and Four-axis Maturity Driven DevSecOps (MFAM) model developed to measure security improvements by design in the CI pipeline. The implementation and validation of this model were done in two methods. Firstly, the metrics developed in the model are validated through the use of standards such as OWASP SAMM and ISO27001 2017. Secondly, it is checked to see if the model can be used in practice to measure improvements using a case study on the company BiZZdesign. For this contextual investigation, applications in BiZZdesign CI pipelines were measured. The measurement came in two phases. In phase one, the measurement of the state of existing systems at BiZZdesign is done, and in phase two, The addition of SonarCloud(SAST) and OWASP ZAP (DAST) is done. Finally, a comparison and evaluation of the two phases are made to estimate the level of improvement in the CI pipeline. Several observations and key takeaways were then made towards the ending of the thesis. First, the quantification of how much a system improved when security tools were added was achieved. The MFAM model detected the changes in the maturity of security when security tools were added as it showed an increase of one or two levels in each axis of the MFAM model. However, factors like agility, productivity, and integration methods all came as points that can affect the addition of security as a part and body of the DevOps cycle. As much as security improvements were seen through the model, the scope is limited as the model is only used in one case study