TCP adaptation for MPI on long-and-fat networks

Typical MPI applications work in phases of computa-tion and communication, and messages are exchanged in relatively small chunks. This behavior is not optimal for TCP because TCP is designed only to handle a contigu-ous flow of messages efficiently. This behavior anomaly is well-known, but fixes are not integrated into today’s TCP implementations, even though performance is seriously de-graded, especially for MPI applications. This paper pro-poses three improvements in the Linux TCP stack: i.e., pac-ing at start-up, reducing Retransmit-Timeout time, and TCP parameter switching at the transition of computation phases in an MPI application. Evaluation of these improvements using the NAS Parallel Benchmarks shows that the BT, CG, IS, and SP benchmarks achieved 10 to 30 percent improve-ments. On the other hand, the FT and MG benchmarks showed no improvement because they have the steady com-munication that TCP assumes, and the LU benchmark be-came slightly worse because it has very little communica-tion.