Understanding Torus Network Performance through Simulations

Abstract-- Technology developments in the storage and processing of data have spurred the development of distributed computing with distributed compute-clusters and supercomputers processing massive data that typically accompanies scientific experiments in the sciences. This has led to increasing demands for data transfers, with a requirement for high speed as well as requirements for Quality of Service, reliability, and security. These issues become more important in high-speed networks. A major consideration in the design of any parallel systems is the set of pathways over which the nodes communicate with each other. A Torus interconnect is a network topology for connecting processing nodes in a parallel computer system. A number of supercomputers on the TOP500 list use 3D Torus networks. In this work we benchmark the Torus network through appropriate performance metrics under different workloads using the ROSS(Rensselaer’s Optimistic Simulation System)simulator. ROSS is a parallel discrete-event simulator that executes on shared-memory multiprocessor systems which is geared for running large-scale simulation models (i.e., supporting millions of object models is feasible). Through synthetic benchmarks, we have studied the communication imbalance generated by the common static single path routing in Torus interconnects. The long term goals are to demonstrate that multi-path dynamic routing could have significantly positive impact on both the end-to-end application performance as well as the aggregate system wide performance. I