Analyzing Sampling Stability of Highly Configurable Systems: Project Work

The variability of software product lines is a significant challenge to efficient software testing. Therefore, strategies such as sample generation reduce the testing effort. T-Wise Combinatorial Interaction Testing (CIT) has proven to be a useful sampling strategy in the context of software product lines. CIT sampling algorithms compute a set of configurations that covers a given degree of feature interactions. The error detection rate depends on the degree of t-wise coverage achieved. We study two methods that consider the evolution of software product lines to increase the fault detection rate. Continuous T-Wise Coverage offers the possibility to achieve higher t-wise coverage in the context of continuous integration. Sampling stability indicates how similar the samples are that a sampling algorithm generates over the evolution of a software product line. We implement the sampling stability and continuous t-wise coverage methods in the existing e4CompareFramework. For the evaluation, we use several real-world systems, which have a sufficiently large evolution. However, the deficient availability of feature models for these systems is a challenge. We present a method to extract feature models from real-world systems configured with the Kconfig language