RAPIER: Integrating Routing and Scheduling for Coflow-aware Data Center Networks

Abstract—In the data flow models of today’s data center applications such as MapReduce, Spark and Dryad, multiple flows can comprise a coflow group semantically. Only completing all flows in a coflow is meaningful to an application. To optimize application performance, routing and scheduling must be jointly considered at the level of a coflow rather than individual flows. However, prior solutions have significant limitation: they only consider scheduling, which is insufficient. To this end, we present RAPIER, a coflow-aware network optimization framework that seamlessly integrates routing and scheduling for better application performance. Using a small-scale testbed implementation and large-scale simulations, we demonstrate that RAPIER significantly reduces the average coflow completion time (CCT) by up to 79.30 % compared to the state-of-the-art scheduling-only solution, and it is readily implementable with existing commodity switches. I