Expoiting Spatial Parallelism in Ethernet-based Cluster Interconnects

In this work we examine the implications of building a single logical link out of multiple physical links. We use MultiEdge [12] to examine the throughput-CPU utiliza-tion tradeoffs and examine how overheads and performance scale with the number and speed of links. We use low-level instrumentation to understand associated overheads, we experiment with setups between 1 and 8 1-GBit/s links, and we contrast our results with a single 10-GBit/s link. We find that: (a) Our base protocol achieves up-to 65 % of the nominal aggregate throughput. (b) Replacing the in-terrupts with polling significantly impacts only the multiple link configurations, reaching 80 % of nominal throughput. (c) The impact of copying on CPU overhead is significant, and removing copying results in up-to 66 % improvement in maximum throughput, reaching almost 100 % of the nomi-nal throughput. (d) Scheduling packets over heterogeneous links requires simple but dynamic scheduling to account for different link speeds and varying load. 1