Ensuring Fairness Among Participating Clusters During Multi-site Parallel Job Scheduling

Multi-cluster schedulers can dramatically improve average job turn-around time performance by making use of fragmented node resources available throughout the grid. By carefully mapping job’s across potentially many clusters, jobs that would otherwise wait in the queue for local cluster resources can begin execution much earlier; thereby improving system utilization and reducing average queue waiting time. In this paper, we demonstrate that these multi-site scheduling techniques can be successfully integrated with fairness policies to ensure that participation in the multicluster is beneficial under extremely disparate workload intensities. Furthermore, we demonstrate that the trade-off between fairness and performance is relatively small. 1