Evaluating Budget of Overhead and Scalability on High-Performance Computing Systems

ABSTRACT. High-Performance Computing Systems (HPCS) based on parallel processing have the potential for satisfying the rapid growth in computational requirements at economical costs. As such machines grow in size, however, parallelization overhead grows in a manner that can limit scalability. This work sheds some light on the sources, dynamics, and magnitudes of the different types of overhead and their impact on performance. Results are obtained through experimental measurements of one of the NASA Earth and Space Sciences (ESS) applications running on the Jet Propulsion Laboratory/Earth and Space Sciences High-Performance Computing (HPC) systems