An empirical study to evaluate a domain specific language for formalizing software engineering experiments

The research about the formalization and conduction of controlled experiments in software engineering has reported important insights and guidelines for conducting an experiment. However, the computational support to formalize and execute controlled experiments still requires deeper investigation. In this context, this paper presents an empirical study that evaluates a domain-specific language proposed to formalize controlled experiments in software engineering. The language is part of an approach that allows the generation of executable workflows for the experiment participants, according to the statistical design of the experiment. Our study involves the modeling of eight software engineering experiments to analyze the completeness and expressiveness of our domain-specific language when specifying different controlled experiments. The results highlighted several limitations that affect the formalization and execution of experiments. These outcomes were used to extend the evaluated domain-specific language. Finally, the improved version of the language was used to model the same experiments to show the benefits of the improvements