The Variability-Space Topography of Interaction Faults

As software grows more prevalent and more integral to human activities, the potential impact of faulty software also grows, making cost-effective anticipation, prevention, and mitigation of software failures all the more important. But the rising number of purposes to which software is put also drives systems to become increasingly versatile and thus larger and more configurable. Once a system becomes highly configurable, the challenges of verification become subject to a combinatorial explosion: software engineers must consider not only whether each individual feature satisfies its requirements, but also whether requirements are satisfied under every feasible combination of those features, as the features might interact. While large combinatorial spaces can be dealt with, any such treatment depends critically on an accurate model of how faults are tied to the system configuration. Unfortunately, existing techniques for characterizing behaviorial properties like faultiness in highly configurable software use models that do little to account for the differences between a system\u27s features or the ways in which those features are related to each other. For instance, the most popular approach, fixed-strength combinatorial interaction testing, relies on estimates of the number of features that a fault might involve while ignoring questions about which features might be counted in those estimates. This imprecision limits practitioners\u27 ability to focus on fault-prone interactions, making it more difficult to prevent, diagnose, or mitigate interaction faults, and it also limits researchers\u27 ability to discern patterns in faults that might inform improved practices. In this work I develop a formalism, variability-space topography, with which one can express more nuanced relationships between features and configuration-dependent properties like the presence of a fault. I then apply variability-space topography to real systems in three studies using a dynamic analysis, a static analysis, and a combined dynamic/static analysis, and find that the additional precision reveals a structure underlying the distribution of interaction faults. I also demonstrate the practical value of these findings in applications such as mapping configuration-dependent code, avoiding failures by reconfiguration, distinguishing interaction and non-interaction faults, and seeding realistic configuration-dependent and interaction faults