Generating Optimized Samples with Attributed Feature Models: Master's Thesis

Sample generation is a powerful and common strategy to address the challenges of testing highly configurable systems efficiently. In the context of software product lines, t-wise Combinatorial Interaction Testing (CIT) has proven to be an effective sampling strategy to minimize the number of test cases. In recent years, researchers have designed novel t-wise sampling algorithms that are becoming more efficient and scaling with larger software product lines. However, in practice, test configurations are often created randomly or manually with expert knowledge. To automate the selection of test configurations based on expert knowledge, the feature models can be extended with expert knowledge in the form of attributes. The selection of test configurations frequently becomes a multi-dimensional problem, as the test configurations are intended to achieve different conflicting objectives, such as t-wise feature interaction coverage, minimization of test cases, maximization of the selection of optional features, or optimization of an attribute. Researchers have developed various approaches for multi-objective test generation. One disadvantage of those is the trade-off between t-wise feature interaction coverage and attribute optimization. In this master thesis, we present two t-wise sampling algorithms that cover all t-wise feature interactions and optimize the sample based on selected attributes of the attributed feature model. For this, we define the meaning of sample optimization based on feature attributes and design a metric to evaluate optimization effectiveness. For evaluating the efficiency and effectiveness of our sampling algorithms, we compare them with other state-of-the-art t-wise sampling algorithms. For this purpose, we use attributed feature models of real-world systems. Some of them are artificially enriched with attributes