Effects of Communication Latency, Overhead, and Bandwidth in a Cluster Architecture. Computer architecture news

This work provides a systematic study of the impact of commu-nication performance on parallel applications ina high performance network of workstations. We develop an experimental system in which the communication latency, overhead, and bandwidth can be independently varied to observe the effects on a wide range of ap-plications. Our results indicate that current efforts to improve clus-ter communication performance to that of tightly integrated paral-lel machines results in significantly improved application perfor-mance. We show that applications demonstrate strong sensitivity to overhead, slowing down by a factor of 60 on 32 processors when overhead is increased from 3 to 103,us. Applications in this study are also sensitive to per-message bandwidth, but are surprisingly tolerant of increased latency and lower per-byte bandwidth. Fi-nally, most applications demonstrate a highly linear dependence to both overhead and per-message bandwidth, indicating that further improvements in communication performance will continue to im-prove application performance.