Design and implementation of a high performance user level network interface for cluster computing

The goal of this project is to study the design and implementation of a new intelligent network interface for improved performance of distributed problems oil workstation clusters. By making use of recent advances in embedded inicrocontroller technology and field programmable gate arrays, a new network interface was able to be constructed that offloads many of the essential, time consuming tasks associated with messaging traffic from the main CPU, thereby reducing the overhead incurred by inter-node communications. This allows the application-to-application latency to be reduced significantly, which allows a broader range of problems to be profitably attacked on workstation clusters. Studies have demonstrated that reducing communication overhead on both the main CPU and any provided network processor can have a large impact on overall system performance. Additionally, the use of highly integrated embedded processors allows for both an extremely capable and cost-effective implementation of such an interface. By providing the network interface with sufficient resources to allow it to perform many of the tasks that previously could only be handled by the operating system, the host processor is free to spend more of its time on useful application processing. However, the offloading of such tasks requires that the host operating system and the network interface be carefully integrated to allow them to work together efficiently. To demonstrate the feasibility of such a design, a prototype was implemented and tested, which allowed for direct, application-to-application communication in as little as 5.90 microseconds, with very little incurred main processor overhead. Additionally, the prototype was demonstrated to provide good performance on a variety of benchmark applications. The hope is that by demonstrating that for minimal additional expense, more capable network interface designs can offer significant performance gains, which would hopefully encourage manufacturers to consider them as additions for future generations of products