Exploring fine-grained task-based execution on multi-gpu systems

Using multi-GPU systems, including GPU clusters, is gaining popularity in scientific computing. However, when using multiple GPUs concurrently, the conventional data parallel GPU programming paradigms, e.g., CUDA, cannot satisfactorily address certain issues, such as load balancing, GPU resource utilization, overlapping fine-grained computation with communication, etc. In this pa-per, we present a fine-grained task-based execution frame-work for multi-GPU systems. By scheduling finer-grained tasks than what is supported in the conventional CUDA programming method among multiple GPUs, and allow-ing concurrent task execution on a single GPU, our frame-work provides means for solving the above issues and ef-ficiently utilizing multi-GPU systems. Experiments with a molecular dynamics application show that, for non-uniform distributed workload, the solutions based on our framework achieve good load balance, and considerable performance improvement over other solutions based on the standard CUDA programming methodologies