Firmware Support for Reliable Communication and Dynamic System Configuration in System Area Networks

Research efforts to provide fast multicomputer interconnection networks for clusters of workstations (SANs) at low cost has lead to the design of network interfaces that support minimal standard functionality in the network interface and use powerful basic blocks that allow communication system designers to implement the functionality needed by different paradigms and systems using these basic blocks. These network interfaces usually encompass a general purpose processor, use simple switches to route packets to their destination, and allow for arbitrary network topologies. These characteristics result in the absence of reliability in the network interface hardware and the need to support routing for arbitrary, dynamic topologies in the communication system. There are many ways to deal with these problems. However, if performance is not to be compromised, these problems are highly non-trivial and many issues need to be considered. Data copies need to be avoided, algorithms and protocols need to be simple and memory requirements must be low, since the network interface has limitations in both network processor speed and memory. In this work we address these two issues by presenting the extensions we did to our communication system, called Virtual Memory Mapped Communication (VMMC), to support reliable communication in the network interface and dynamic system configuration. We propose and implement reliable communication in the firmware that runs on the network interface processor with minimal cost; less than 2.5 s for latency and less than 10 % for all different types of bandwidth Moreover, we propose a new scheme for dynamically determining the network topology. This mechanism relies on the retransmission mechanism that is used to provide reliable communication. Since the support for dynamic system configuration is not on the common path, there is no effect on system performance when no changes in the system topology occur.