Effects of Communication Latency, Overhead, and Bandwidth in a Cluster Architecture. Computer architecture news

This work provides a systematic study of the impact of commu-nication performance on parallel applications in a high performance network of workstations. We develop an experimental system in which the communication latency, overhead, and bandwidth can be independently varied to observe the effects on a wide range of ap-plications. Our results indicate that current efforts to improve clus-ter communication performance to that of tightly integrated paral-lel machines results in significantly improved application perfor-mance. We show that applications demonstrate strong sensitivity to overhead, slowing down by a factor of 60 on 32 processors when overhead is increased from 3 to 103 s. Applications in this study are also sensitive to per-message bandwidth, but are surprisingly tolerant of increased latency and lower per-byte bandwidth. Fi-nally, most applications demonstrate a highly linear dependence to both overhead and per-message bandwidth, indicating that further improvements in communication performance will continue to im-prove application performance.