Understanding Software Development and Testing Practices

Graduation date: 2017A bad software development process leads to wasted effort and inferior products. In order to improve a software process, it must be first understood. In this work I focus on understanding software processes.\ud The first process we seek to understand is Continuous Integration (CI). CI systems automate the compilation, building, and testing of software. Despite CI rising as a big success story in automated software engineering, it has received almost no attention from the research community. For example, how widely is CI used in practice, and what are some costs and benefits associated with CI? Without answering such questions, developers, tool builders, and researchers make decisions based on folklore instead of data.\ud We use three complementary methods to study the usage of CI in open-source projects. To understand which CI systems developers use, we analyzed 34,544 open-source projects from GitHub. To understand how developers use CI, we analyzed 1,529,291 builds from the most commonly used CI system. To understand why projects use or do not use CI, we surveyed 442 developers. With this data, we answered several key questions related to the usage, costs, and benefits of CI. Among our results, we show evidence that supports the claim that CI helps projects release more often, that CI is widely adopted by the most popular projects, as well as finding that the overall percentage of projects using CI continues to grow, making it important and timely to focus more research on CI.\ud Furthermore, we present a qualitative study of the barriers and needs developers face when using CI. In this paper, we conduct 16 semi-structured interviews with developers from different industries and development scales. We triangulate our findings by running two surveys. The Focused Survey samples 51 developers at a single company. The Broad Survey samples a population of 523 developers from all over the world. We identify trade-offs developers face when using and implementing CI. Developers face trade-offs between speed and certainty (Assurance), between better access and information security (Security), and between more configuration options and better ease of use (Flexibility). We present implications of these trade-offs for developers, tool builders, and researchers.\ud Additionally, we seek to use code and test changes to understand conformance to the Test Driven Development (TDD) process. We designed and implemented TDDViz, a tool that supports developers in better understanding how they conform to TDD. TDDViz supports this understanding by providing novel visualizations of developers’ TDD process. To enable TDDViz’s visualizations, we developed a novel automatic inferencer that identifies the phases that make up the TDD process solely based on code and test changes.\ud We evaluate TDDViz using two complementary methods: a controlled experiment with 35 participants to evaluate the visualization, and a case study with 2601 TDD Sessions to evaluate the inference algorithm. The controlled experiment shows that, in comparison to existing visualizations, participants performed significantly better when using TDDViz to answer questions about code evolution. In addition, the case study shows that the inferencing algorithm in TDDViz infers TDD phases with an accuracy (F-measure) of 87%