Design and investigation of scheduling mechanisms on accelerator-based heterogeneous computing systems

Modern high-performance computers engage a variety of computing devices. Underutilization and oversubscription of processing resources are the common problems in modern accelerator-based computing systems. Particularly, if different tasks are processed simultaneously, achieving a good level of both resource utilization and fairness at the same time is not trivial. For this purpose and to simplify the use of accelerators and to improve their usability while sharing them among multiple users, this thesis explores several design factors of various modern processing architectures and investigates task-scheduling on accelerator-based heterogeneous computing systems. First of all, using diverse microbenchmark suites and real-world applications from the field of physics, the runtime behavior of test programs is observed. Major architectural discrepancies are studied and a higher-level examination is discussed, involving the single unified programming interface OpenCL. Subsequently, starting from the results of this investigation, two independent task-scheduling approaches for heterogeneous systems are pursued. Each scheduling framework considers a certain group of processing architectures, depending on the underlying programming interface. The design characteristics of both frameworks differentiate according to the architecture of the managed devices on the one hand, as well as the corresponding programming framework -- CUDA or OpenCL -- on the other. While GPUSched is designed and implemented for managing the execution of CUDA applications on multiple NVIDIA GPUs, the focus on heterogeneity represents the contribution of the OpenCL-based scheduling framework OCLSched discussed in the second approach. However, in both cases programmers formulate their own tasks in the corresponding programming language, whereby the scheduler autonomously takes over and manages the processing of applications in a heterogeneous computing environment. Subsequently, the parallelization of a modern algorithm for similarity search in multimedia databases is discussed, considering multiple performance- and efficiency issues. With the presented scheduler, the similarity search algorithm is processed with general computation tasks in a multi-tenant compute environment simultaneously. During the experiments, the extended functionality and the deployed strategies on the one hand, and the scheduling algorithms on the other are discussed and evaluated. It will be shown that by means of concurrent kernel execution and multitasking, the presented scheduler achieves better resource utilization, although the job execution times are not specified in advance. Through the headway in understanding the architectural characteristics of modern accelerators in depth on the one hand, and the promising results of the presented scheduling mechanisms on the other, this thesis contributes to the innovation of efficient heterogeneous computing clusters and describes highly-optimized programming interfaces designed for accelerator-based computers