Towards scalable implicit communication and synchronization: Explicit concurrency control, unscoped synchronization and implicit data dependencies considered harmful

There exists several divides between implicit and explicit paradigms in concurrent programming models, for example between the assumption of coherent shared memory (e.g. OpenMP), and the assumption of distributed memory (e.g. MPI). Explicit paradigms exist to provide control to programmers, but cause scalability concerns: programs need to be adapted whenever the granularity of concurrency changes. With the rise of large heterogeneous pools of computing resources, we must increasingly distribute tasks automatically. Implicit paradigms allow this in theory and are desirable for expressivity and intuitiveness, but their scalability in heterogeneous environments is yet unclear. In this position paper, we propose to consolidate previous knowledge by seeking more implicit concurrent programming models that combine three properties. The first desirable property is resource agnosticism, where programs separate clearly the description of computations from the description of task distribution to resources. The second property is scoped synchronization, where programs express no more synchronization than required by the described computation. The third property is the visibility of data dependencies between tasks by compilers and run-time systems. Only when these properties exist together, it becomes possible to automatically tailor programs to heterogeneous target systems and achieve efficient execution. We show how specializability is needed to optimize this process