An empirical study of specification by example in a software engineering tool

Meta-CASE tools offer CASE tool specialisation by enabling a designer to specify a tool which is then generated automatically. Constraints are often used in such meta-CASE tools for governing the syntax and semantics of model elements and the values of their attributes. However, the constraint definition process is complex, time-consuming and error-prone. This paper presents an empirical study of the use of Specification by Example (SBE), based on the well-known notion of Programming by Example (PBE), as a user-computer interactive technique for such constraint specification. Two constraint specification techniques have been implemented in a meta-CASE tool a wizard that represents a conventional form-filling technique and an SBE technique that depends on the user providing one or more examples and the system inferring a list of possible intended constraints. The empirical study compared the wizard and SBE with respect to constraint definition correctness, task completion time, and user satisfaction. Two common modelling diagrams have been used, a State Transition Diagram and a Use Case Diagram. Results suggest that SBE is superior to the wizard in terms of measured criteria described above. Observations on the interaction of users with the system and opinions of participants are also presented