Assessment framework for a methodology under development – Application to the PDA methodology

This paper presents an assessment framework for methodologies under development. It is adapted from the evaluation framework for the design of an engineering model developed by Ben Ahmed and colleagues [1]. The assessment framework allows to take into account in a systematic way characteristics (that is, main categories or classes of potential requirements) that are of importance for the assessment of the quality a methodology, beyond effectiveness and efficiency. The framework is intended to be employed similarly to engineering design requirements checklists: ensuring first that no important characteristics are left out; deriving from these characteristics more specific requirements when necessary; using these characteristics or the derived requirements both to drive the development of the methodology and as evaluation criteria to assess the elements of the developed methodology. These characteristics can then be screened again as the methodology is improved. As the methodology is being developed, the assessments can go from wide and qualitative to more stringent. The framework is applied to assess the predictive design analysis (PDA) methodology.[1] Ben Ahmed, W., Mekhilef, M., Yannou, B., and Bigand, M., (2010), "Evaluation framework for the design of an engineering model", Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 24(1), pp. 107-125