HDM Selection of Emerging Technology for Supercomputing Storage

Choosing to move to new technologies is a complex process, and many decisions must be made when choosing the technologies for the next supercomputer. This paper identifies key criteria for a national lab choosing storage technologies for the next generation of supercomputing and offers a model for determining one of the decisions: which emerging technology should be used for long term storage on a supercomputer for AI workloads. This paper describes these types of emerging storage technologies as well as presents a Hierarchical Decision Model (HDM) structure for choosing a storage technology strategy. The authors create a Hierarchical Decision Model (HDM) model which may be used as a basis for future storage technology decision making at various national laboratories with different workloads. This implementation of the model has been designed for a generic national laboratory, and the pairwise comparison judgments are based solely upon interviews around supercomputing and literature review about the technologies. The Storage HDM model was developed using four levels of criteria. The first level, the Mission Level, was crafted to be “Determine the emerging technologies to be pursued in the next generation of supercomputing at USA national laboratories for AI Workloads”. The second level, Objective Level, criteria were gathered from the literature review and expert opinion. The three objectives are Technological Form Factors, Economic, and Policy/Politics. To limit the scope of this project as well as keep the expert pairwise comparison data points manageable; the team focused on two criteria per objective on the third level except for Technology. The technical objective has “Reliability”, “Durability”, “Workload Bandwidth”, and “Data Storage Capacity” The Economic objective has “Production Cost” and “Productization Ability.” Finally, the Policy/Politics objective has “Drive New Technology” and “Novel Technology compared to other labs.” The last level contains the seven emerging storage technology strategies which the HDM is comparing. The seven strategies are Hybrid Cloud HPC Storage, HAMR: Heat-assisted Magnetic Recording, Helium Storage, DNA Storage, Magnetic Recording, NVM: Non-volatile Memory, and HDD. We removed Hybrid Cloud HPC Storage as it wasn’t a viable candidate for supercomputing - as it reduces the security and places compute power at the variable demands of network bandwidth. The HDM model evaluation used a single expert panel. The experts evaluated the priorities of the objectives and criteria in an aim to select the best digital storage device method. The similar platter based storage systems: HAMR, HDD, Helium, Magnetic Recording had similar outcomes. DNA data storage and NVMe came out very close as the top choices. Additional research would need to be done as DNA storage becomes a more viable and financially realistic solution