Evaluating an Emulation Environment: Automation and Significant Key Characteristics: Paper - iPRES 2012 - Digital Curation Institute, iSchool, Toronto

Evaluating digital preservation actions performed on digital objects is essential, both during the planning as well as quality assurance and re-use phases to determine their authenticity. While migration results are usually validated by comparing object properties from before and after the migration, the task is more complex: as any digital object becomes an information object only in a rendering environment, the evaluation has to happen at a rendering level for validating its faithfulness. This is basically identical to the situation of evaluating the performance in an emulation setting. In this paper we show how previous conceptual work is applied to an existing emulator, allowing us to feed automated input to the emulation environment as well as extract properties about the rendering process. We identify various significant key characteristics that help us evaluate deviations in the emulator’s internal timing compared to the original system and how we can find out if the emulation environment works deterministically, an important characteristic that is necessary for successful comparison of renderings. We show the results of rendering different digital objects in the emulator and interpret them for the rendering process, showing weaknesses in the evaluated emulator and provide possible corrections as well as generalized recommendations for developing emulators for digital preservation