Co-location of Deep Learning Jobs in GPU Clusters

Deep learning (DL) training jobs now constitute a large portion of the jobs in the GPU clusters. Following the success of deep learning in various domains such as natural language processing, image classification, and object detection GPUs have become the new member of the computing clusters. Due to various reasons, GPUs are highly underutilized in the production GPU clusters. In this thesis, we design a scheduler that uses co-location to improve the GPU utilization in these clusters. Using in-depth profiling of DL jobs, we provide metrics that guide us on the compatibility of different DL jobs. Using these profiling data we are able to achieve almost 2X speedup in the makespan when using co-location compared to the first-in-first-out baseline.M.A.S