Implementation of processor farms on myrinet network interface cards

Processor farms are an intuitive way of parallelizing a program in which a manager processor farms out tasks to a set of worker processors that process the tasks and send the results back to the manager. In this thesis we consider processor farms as a possible intermediate level model which is sufficiently high level to be easy to use, but simple enough to run efficiently on the latest cluster architecture. Advances in programmable interconnects have made it possible to increase the efficiency of processor farms. These interconnections have a programmable processor which enables us to offload the communication to the boards. Also, due to the recent rapid increase in processor speeds, using high speed interconnections allows the communication to keep up with the computation. The experimental test bed for our processor farm system is a cluster of sixteen Pentium II computers interconnected by a Myrinet [18] gigabit per second programmable network. We developed and optimized a processor farm, which we call pfarm for our sixteen node cluster. We show that pfarm effectively executes relatively fine grain tasks with variable execution times, using a programmable network interface card to almost completely overlap communication and computation.Science, Faculty ofComputer Science, Department ofGraduat