Adaptive Message Pipelining for Network Memory and Network Storage

Recent advances in cluster file systems, network memory, and network-attached disks make it possible to construct storage systems whose performance tracks network technology rather than disk technology. However, delivering the potential of high-speed networks for network storage systems depends on communication support that meets client demands for low network latency for random block accesses and high bandwidth for streaming block accesses. This paper describes an adaptive message pipelining technique that reduces network latency of messages larger than 1 KB while delivering the full bandwidth of gigabit-per-second networks and PCI I/O buses. We report on experiences with the Trapeze messaging system, which implements adaptive message pipelining in firmware for a Myrinet network interface. We show that the Trapeze approach is superior to other message pipelining alternatives, and illustrate its benefits with application experiments using the Global Memory Service, a network memory sys..