Improving Hardware Multithreading in General Purpose Graphics Processing Units

General-purpose graphics processing unit (GPGPU) is one of the most popular many-core acceleratorsthat deliver a massive computing power in parallel applications. GPGPUs mainlyrely on the hardware multithreading to hide a short pipeline stall and a long memory latency.Thus, the performance of GPGPU can be signicantly aected by how GPGPU'shardware multithreading is applied. However, nding the optimal hardware multithreadingis a complex problem since there are many aspects to be considered. This work studies themechanisms for improving the eectiveness of hardware multithreading. First, it studiesthe various scheduling policies and proposes an adaptive scheduling policy that chooses thebest scheduling policy at runtime. In addition, it proposes simple but eective warp throttlingmechanism that can increase the cache locality. Furthermore, it proposes a hardwareprefetching mechanism to extend the memory latency hiding degree of hardware multithreading.Finally, it shows how a limited scalability of the conventional cache miss handling architectureconstrains the degree of hardware multithreading and proposes the highly scalablecache miss handling architecture