Exploring the Predictability of MPI Messages

Scalability to a large number of processes is one of the weaknesses of current MPI implementations. Standard implementations are able to scale to hundreds of nodes, but no beyond that. The main problem of current implementations is that performance is more important than scalability and thus some assumptions about resources are taken that will not scale well. The objective of this paper is twofold. On one hand, we show that characteristics such as the size and the sender of MPI messages are very predictable (accuracy above 90%). On the other hand, we present some examples where current MPI implementations would not work well when run on a large configuration and how this predictability could be used to solve the scalability problem