Direct-dependency-based software compatibility testing

Software compatibility testing is an important quality assur-ance task aimed at ensuring that component-based software systems build and/or execute properly across a broad range of user system congurations. Because each conguration can involve multiple components with dierent versions, and because there are complex and changing interdependencies between components and their versions, it is generally in-feasible to test all potential congurations. Therefore, com-patibility testing usually means examining only a handful of default or popular congurations to detect problems, and as a result costly errors can and do escape to the eld. This paper presents an improved approach to compati-bility testing called RACHET. We formally model the con-guration space for component-based systems and use the model to generate test plans covering user-specied portion of the space { the example in this paper is covering all direct dependencies between components. The test plan is exe-cuted eciently in parallel, by distributing work so as to best utilize test resources. We conducted experiments and simulation studies applying our approach to a large-scale data management middleware system. The results showed that for this system RACHET discovered incompatibilities between components at a small fraction of the cost for ex-haustive testing without compromising test quality