Evaluation of a Pattern Language for JHotDraw

Pattern languages are widely considered to be a useful form of framework documentation without ever having been scrutinised by an objective evaluation. This lack of validation prevents framework developers from understanding why pattern languages work and how to codify their construction so that other developers can also write useful pattern languages for their systems. Without concrete evidence of the effectiveness of pattern language techniques the framework community should be more sceptical of them as a valid form of framework documentation. If research in this area does not become more through, specifically by validating the claims made by supporting documentation, then framework reuse may never overcome its large communication overhead and make a significant impact on the future of software engineering. This paper documents an attempt to provide an objective analysis of the effectiveness of pattern language documentation. It analyses the experiences of final year undergraduate students faced with the task of reusing the JHotDraw application framework. The investigation took place over an eight-week period and used a multi method approach to capture as rich a dataset as possible. The results show that pattern languages are a useful form of framework documentation but raise questions over the assumptions made in the design of current pattern languages particularly about the merger of patterns with other complementary forms of documentation. This paper also describes the difficulties in collecting meaningful empirical data about documentation effectiveness to assist future developers to perform more effective comparisons in the future. 2