On design-oriented research and digital learning materials in higher education

The context of the research described in this thesis is formed by a number of research projects that were aimed at the design, development, implementation, use and evaluation of innovative digital learning materials. Most of these projects were carried out mainly within Wageningen University. In this thesis, these projects are collectively referred to as 'WU Projects'. During this research it became clear that available literature provided insufficient support with respect to a number of issues. Examples are 'How to phrase research questions?', 'What output to expect?', 'What type of evaluation is relevant?' and 'What methods should be used?'. In fact, in parallel with the WU projects, the body of literature on methodology for design related research approaches in several disciplines was growing considerably. This thesis aims to contribute to this methodological discussion. In addition, this thesis presents a view on the characteristics and possibilities of digital learning materials in higher education. In Chapters 2,3 and 4, a methodological framework for design, development, implementation, use and evaluation of innovative digital learning materials in higher education is defined and elaborated. Research that fits this framework is called design-oriented research (DOR). The framework is the result of a systems-oriented theoretical discussion of literature from a range of knowledge domains such as learning and instruction, knowledge and information systems research and engineering design. The concepts and terminology are illustrated with examples from publications that resulted from various WU projects. In addition, part of the framework is captured in a glossary of terms. The glossary aims to provide a coherent and ‘workable’ set of terms and corresponding definitions or descriptions. For many terms, this implies a compromise between natural language preferences of members of different disciplines. In Chapter 4, a classification of outputs that are potentially valuable is presented. An important implication of the view presented in this chapter, is that the actual design goal can be output, rather than input of a DOR project. This is in agreement with the view of instructional design and design of digital learning resources as processes of constraint exploration and constraint satisfaction. In Chapter 5 and Chapter 6 the most important strategic decisions in DOR projects that aim to deliver digital learning materials in higher education are discussed. In Chapter 5, a classification of design goals with their relation to various knowledge domains is presented. In Chapter 6, a classification of large-scale use scenarios with their relation to design requirements is given. Finally, Chapter 7 describes requirements and opportunities for evaluation in design-oriented research in education and reflects on evaluation in a number of WU projects. This leads to a number of suggestions for improvement with respect to evaluation in DOR.