Empirical Comparison of Language Workbenches

Production use of Domain-Specific Modeling languages has consistently shown productivity increase by a factor of 5–10. However, the spread of DSM has been slowed by projects stalling even before the language was built, often citing problems with the chosen tool. With a wide variety of language workbench tools for DSM, there is a need for objective empirical tool comparison – particularly as the little research so far shows a range of 50 times more effort between the most and least efficient tools. This article looks at existing empirical research and an experimental design for a future comparison. We aim at increasing objectivity and repeatability while keeping overall effort practical, and providing worthwhile returns for the participants