Enabling Efficient Resource Utilization on Multitasking Throughput Processors.

Graphics processing units (GPUs) are increasingly adopted in modern computer systems beyond their traditional role of processing graphics to accelerating data-parallel applications. The single instruction multiple thread (SIMT) programming model enabled programmers to easily offload data-parallel kernels to the GPUs, and achieve large performance improvements and energy efficiency. As a result, many supercomputers, cloud services, and data centers are utilizing GPUs as general-purpose throughput processors. In these modern computer systems, achieving high resource utilization becomes important, especially in the shared environment like cloud services, and data centers. To enable efficient resource utilization of multitasking GPUs, new unique challenges like enabling an efficient preemptive multitasking and resolving shared resource contention have to be solved. Many traditional policies such as context switching or fine-grained multitasking can incur a large overhead if blindly applied. To address these challenges with minimal overhead, a framework that exploits both static characteristics of the SIMT programming model and runtime properties of GPU execution is required. This thesis proposes a framework with hardware/software extensions to enable efficient resource utilization on multitasking GPUs. The framework identifies characteristics coming from the SIMT programming model or determines the best policy based on multiple candidates either at compile time or using runtime software. The framework is adaptive and dynamic because it gathers runtime statistics using hardware performance counters to guide the decisions in the runtime software, and implements individual mechanisms in hardware with low overhead. The framework consists of three components. The framework uses compiler hints to improve utilization by an average of 11.9% when running memory-intensive kernels alone, which can provide synergistic improvement when multitasking is enabled. The framework further improves the system throughput by an average of 12.2% with a collaborative preemption mechanism for efficient preemptive multitasking, which can handle both latency-sensitive and throughput-oriented applications. This framework implements a dynamic resource management scheme to maximize the utilization by exploiting both spatial multitasking, which partitions resources at the streaming multiprocessor (SM) granularity, and simultaneous multikernel, which partitions resources within the SM. Combining all the techniques in the framework, 45.0% better system throughput was achieved over single kernel execution.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/133510/1/jasonjk_1.pd