Cost/performance tradeoffs in network interconnects for clusters of commodity PCs

This is an extension of a similar paper from Workshop on Communication Architecture for Clusters (CAC ’03) in conjunction with International Parallel and Distributed Processing Symposium (IPDPS), April 2003, Nice, France. The definition of a commodity component is quite obvious when it comes to the PC as a basic compute engine and building block for clusters of PCs. Looking at the options for a more or less per-formant interconnect between those compute nodes it is much less obvious which interconnect still qualifies as commodity and which not. We are trying to answer this question based on an in-depth analysis of a few common more or less expensive in-terconnects on the market. Our measurements and observations are based on the experience of architecting, procuring and in-stalling Xibalba, a 128 node- 192 processor versatile cluster for a variety of research applications in the CS department of ETH Zurich. We define our unique way to measure the performance of an interconnect and use our performance characterization to find the best cost performance point for networks in PC clusters. Since our work is tied to the purchase of a machine at fair mar-ket value we can also reliably comment on cost performance of the four types of interconnects we considered. We analyze the reason for performance and non-performance for different Fast Ethernet architectures with a set of micro-benchmarks and con-clude our study with performance numbers of some applications. Thus, the reader gets an idea about the impact of the intercon-nect on the overall application performance in commodity PC clusters