Experiences with task-based programming using cluster nodes as OpenMP devices

Programming a distributed system, such as a cluster, requires extended use of low-level communication libraries and can often become cumbersome and error prone for the average developer. In this work, we consider each node of a cluster as a separate OpenMP device, able to run code with OpenMP directives in parallel. We make use of the OpenMP device model to provide an easy and intuitive way to program available cluster nodes. Based on that, we examine modifications that were necessary to make existing task-based applications able to exploit such an infrastructure. Finally, we evaluate the performance of the system and conclude that one can gain significant speedup, as long as the application tasks do not produce excessive communication overheads.Comment: In HPCS 2020, The 18th International Conference on High Performance Computing and Simulation, March 202