Developing a holistic model for digital library evaluation

The objective of the research is to develop a holistic model for digital library (DL) evaluation. To develop such a model, a three-stage research approach was applied: exploration, confirmation, and verification. During the exploration stage, a literature review was conducted, and then an interview along with card sorting technique was employed to collect perceptions from DL experts with emphasis on determining what criteria should be used in DL evaluation. Then, the criteria identified from the exploration were used for developing an online survey during the confirmation stage. Heterogeneous DL stakeholders were asked to rate the importance of each criterion to DL evaluation. The holistic model was constructed by utilizing descriptive and inference statistical techniques. Its holistic nature was ensured through: (1) incorporation of various DL stakeholders' perspectives in light of Marchionini's multifaceted evaluation approach, and (2) inclusion of all digital library levels suggested by Saracevic's stratified information retrieval model. Eventually, in the verification stage, selected criteria from the model were tested in real DL use setting. Some significant findings include: (1) consistently perceived important criteria for DL evaluation. DL stakeholders care more about premise (e.g., accessibility and sustainability of a DL), process (e.g., ease of use, technology reliability, and service responsiveness), and direct performance (e.g., usefulness of information, successfulness and efficiency of task completion), whereas less concerned about indirect factors (e.g., personalization, behavior change, service courtesy, and extended social effects); (2) inter-group divergence in importance perception for some evaluation criteria. The divergence primarily exists between the user and other DL stakeholder groups; (3) some promising criteria (e.g., comprehensiveness of collection, integrity of information, integration of service to information seeking path, collaboration/sharing) augment the existing DL evaluations whereby important criteria have essentially been covered; and (4) most importantly, the core dissertation objective is fulfilled, that is the construction of the holistic evaluation model, in which heterogeneous stakeholders' perspectives at all DL levels are presented. The proposed model fills a lacuna in the DL domain, that is the lack of a comprehensive and flexible framework to guide and benchmark evaluations and the uncertainty about what divergence exists among heterogeneous DL stakeholder groups.Ph.D.Includes bibliographical references (p. 183-194)