Improving Distributed Application Performance Using TCP Instrumentation

Distributed scientific computing applications are requiring an ever-increasing amount of bandwidth. Raw network bandwidth has been increasing rapidly, but getting the applications to effectively use this additional bandwidth is becoming more and more difficult. In this paper we describe how TCP instrumentation can be used to guide more efficient use of the network, using a monitoring daemon that we have developed to passively collect TCP instrumentation data from the kernel for a given application. We describe how this daemon was used to analyze GridFTP data flows with some experimental TCP protocol enhancements. One of the primary goals of high performance distributed computing is efficient use of all computing resources, particularly the network. The only way to ensure efficient use of the network is by closely monitoring the behavior of the network protocols such as TCP. In a Grid environment, not only must each network connection be as efficient as possible, but a Grid scheduler or replica selection service must also be able to choose the right set of connections t