A systematical literature review and industrial survey in addressing the possible impacts with the continuous testing and delivery during DevOps transformation.

Context: Digital transformation poses new challenges to the organization. Market needs and competition in business have changed the continuous testing environment and continuous delivery in software organizations. There is a great possibility of conflict between the development and testing operations because they are a separate entity in large organizations. Organizations where testers, developers, and operation teams do not communicate well and lack collaboration can have their productivity affected negatively. This leads to defects and errors at the early stage of the development process. DevOps’ approach enhances the integration, delivery, performance,and communication between the developers, testers, and operational members. Organizations are reluctant to apply DevOps practices because there is a lack of agreement on DevOps characteristics. The most difficult part of a large organization is DevOpsadaptation and its implementation because of its legacy structure. It is required to get an understanding of DevOps implementation in organizations before they start transforming. Objectives: The thesis aims to identify the challenges in organizations towards continuous delivery and provide a list of techniques or strategies to overcome continuous testing and DevOps challenges. This thesis also identifies the communication challenges between continuous testing and delivery teams during the COVID-19 pandemic and the software architecture effect on testing in the DevOps environment. Methods: To achieve the research goal a multiple research method techniques are applied. A systematic literature review is conducted to identify the literature and to meet the research goal. A survey is conducted for the verification of the data from SLR. An interview is used as a data collection method in Survey and explores the actual process of continuous testing and delivery in large DevOps companies. Results: A list of challenges to large organizations towards continuous delivery is generated. A list of strategies and solutions towards the challenges of continuous testing and DevOps is generated. A list of post COVID-19 communication challenges between testing and delivery groups in DevOps is created. A list of software architecture and production environment effects on testing is also generated. After analyzing the SLR results, a survey is conducted to validate the results from software practitioners. Thematic analysis is performed on the results. "Finally", the findings from the SLR and Survey are compared. Conclusions: This research’s findings can help researchers, industry practitioners, and someone who wants to investigate further the possible effects with the continuous testing and delivery during DevOps transformation. We observed that industry practitioners could enhance their communication channels by reviewing the post-COVID-19 communication challenges between testing and delivery teams. We also observed that there is more research required to continue on this topic