Evaluating the benefits of using domain-specific modeling languages: An experience report

There are many tools available for the creation of domain specific languages (DSLs) but the question remains how to identify appropriate use cases for the application of domain specific modeling and language design, and how to measure success. We report on our observations after three years of accompanying several real-life industrial DSL design projects and on our experiments with applying qualitative and quantitative evaluation criteria. We suggest an evaluation methodology spanning the entire DSL life cycle. It consists of an assessment of motivation, qualitative interviews, a validation of DSL design, quantifying benefits and a comparison of impacted workflows before and after adoption. We conclude with a discussion of inherent limitations