MNEMONIC: A Network Environment for Automatic Optimization and Tuning of Data Movement over Advanced Networks

Abstract—Computational science is considered the third pillar of science but while we continue to make significant strides in high-performance computing, the ability to trans-fer the vast amounts of data generated and consumed by scientists has fallen behind. This divergence has signifi-cantly constrained scientific progress. This paper describes MNEMONIC, an end-to-end cyber-infrastructure system designed to improve scientists ’ pro-ductivity by automatically and transparently optimizing and tuning data movement over the network. MNEMONIC integrates advanced technologies, i.e., network interfaces, protocols, and measurement toolkits, to seamlessly manage high-performance data movement. We have integrated the Java implementation of GridFTP in MNEMONIC to pro-vide a cross-platform data transfer capability. Preliminary results show that MNEMONIC+GridFTP outperforms the TCP+GridFTP implementation by over a factor of 10x, while remaining easy to install and operate. I