The impact of type systems and test tooling on codified testing strategies: an exploratory multi-method approach

Software testing has been around for decades and many tools exist to aid developers in their testing process. However, little is known about the rate at which developers test their projects, the tools they use for these purposes and the impact of type systems on testing practices. Our work is the first of its kind to identify and classify available test tooling for programming languages Java, C, JavaScript and Ruby. By conducting a large scale automated analysis on open-source software projects, we show that both available test tooling and type systems have significant impact on codified testing strategies. Our observations are strengthened by targeted interviews and a large-scale survey among developers working with both statically and dynamically typed programming languages. The soft typing system seems like a promising solution, allowing developers to work with the great flexibility less strict type systems provide, while simultaneously benefiting from strict type checks that help reduce the amount of testing required to ensure the correctness of a piece of software. However, future research is needed to estimate the feasibility of such a type system. To help better aid developers in their testing process and to reduce some of the testing burden many developers seem to cope with, we propose a list of test tooling improvements based on our observations