Lightweight Superscalar Task Execution in Distributed Memory

Arguably, we have yet to find a solution to the burden of multicore distributed programming facing domain scientists. This burden has been exacerbated by the increasing size of multicores, increasing the effect of any excess synchronization. To deal with these dif-ficulties, numerical algorithms are re-engineered as sequences of interdependent tile-based tasks which can be executed by a dynamic runtime environment. We present a new runtime environment for distributed architectures which uses superscalar scheduling con-cepts. Tasks are inserted serially, and the runtime determines the dependencies dynamically and manages data movement transpar-ently. QUARK-D (QUeuing and Runtime for Kernels on Distributed Memory) is shown to scale to O(1000) cores for linear algebra al-gorithms and have competitive performance. The primary message of this research is that scalable and competitive performance can be achieved by a distributed-memory execution system using super-scalar scheduling ideas where serial code is the input and parallel execution correctness is guaranteed. 1