Parallelizing the execution of sequential scripts

Scripting is often used in science to create applications via the composition of existing programs. Parallel scripting sys-tems allow the creation of such applications, but each sys-tem introduces the need to adopt a somewhat specialized programming model. We present an alternative scripting approach, AMFS Shell, that lets programmers express par-allel scripting applications via minor extensions to existing sequential scripting languages, such as Bash, and then exe-cute them in-memory on large-scale computers. We define a small set of commands between the scripts and a parallel scripting runtime system, so that programmers can compose their scripts in a familiar scripting language. The underly-ing AMFS implements both collective (fast file movement) and functional (transformation based on content) file man-agement. Tasks are handled by AMFS’s built-in execution engine. AMFS Shell is expressive enough for a wide range of applications, and the framework can run such applications efficiently on large-scale computers