Noise Estimation in HPC Cloud Networks

As computing is moved more and more to the cloud due to scalability, affordability and availability, so are high performance computing (HPC) clusters. It is expensive to build a real-world cluster that needs the scale and performance to run the highest predicted load but is not used to its full extent most of the time. Additionally, by relying on cloud services, the highly specialised workforce needed to maintain the cluster is no longer needed. As a result, more work can be put into the development of applications. With clusters in the cloud, it is possible to build a specific cluster just for the time it is used and with exactly the characteristics that are needed. This significantly lowers the cost of running HPC applications and opens up the possibility to run such applications to a wider variety of researchers and the public. The decreased cost and ease of use of such systems further increase the availability of running HPC workloads. Some leading providers such as Amazon, Microsoft and Google all have their special tailored solutions that try to provide a replication of the performance that a local cluster delivers. Each provider has its solutions to ensure that the nodes that make up the cluster are connected by a consistent high throughput, low latency network and that the virtualisation and sharing of the hardware have no noticeable impact on the performance. But do these promises hold and can it be expected to see the advertised performance on these systems? In this thesis, we start to answer this question. We show differences between the providers and that Azure performs the best and most similar to a non-virtualised cluster