How to reach a usable DSL? Moving toward a Systematic Evaluation

Domain-Specific Languages (DSLs) are claimed to increase productivity,while reducing the required maintenance and programming expertise. In thiscontext, DSL usability by domain experts is a key factor for its successfuladoption.Evidence that support those improvement claims is mostly anecdotal. Oursystematic literature review showed that a usability evaluation was oftenskipped, relaxed, or at least omitted from papers reporting the development of DSLs. Thefew exceptions mostly take place at the end of the development process where fixing problems identified is too expensive.We argue that a systematic approach based on User Interface experimentalvalidation techniques should be used to assess the impact of the new DSLs. The rationale is that assessing important and specially tailored usability attributes for DSLs early in language construction will ultimately foster a higher productivity of the DSL users. This paper, besides discussing thequality criteria, proposes a development and evaluation process that can be usedto achieve usable DSLs in a better way