Moving an Archive from Tape to Disk: A Case-Study at ICPSR

In early 2007 ICPSR's digital archives consisted of 700 magnetic tapes stored in two separate locations in Ann Arbor, Michigan. The best-case scenario for retrieving content consisted of a data librarian finding the correct tape, mounting that tape, restoring the desired content, and then copying that content to a well-known location for the data manager. A worst-case scenario could include trips to off-site storage locations, composing content from a combination of Master and Backup tapes, and multiple attempts at finding just the right content. By late 2007 the archives have been copied to spinning disk, and replicated across storage grids. In true belt-and-suspenders fashion, an additional copy also resides on tape, and this copy of the entire archive fits on a mere six high-density tapes. The online content may be searched and browsed, and with sufficient access rights, an ICPSR data manager may fetch any file through a convenient web interface. This presentation describes the starting point of the migration, challenges faced and lessons learned during the process, and the state of the archives post-migration. We reference technologies that we found useful during this process, but do not probe too deeply into their intricacies