Experimental evaluation of multi-round matrix multiplication on mapreduce

Abstract—This paper proposes an Hadoop library, named M3, for performing dense and sparse matrix multiplication in MapRe-duce. The library features multi-round MapReduce algorithms that allow to tradeoff round number with the amount of data shuffled in each round and the amount of memory required by reduce functions. We claim that multi-round MapReduce algo-rithms are preferable in cloud settings to traditional monolithic algorithms, that is, algorithms requiring just one or two rounds. We perform an extensive experimental evaluation of the M3 library on an in-house cluster and on a cloud provider, aiming at assessing the performance of the library and at comparing the multi-round and monolithic approaches. Keywords—MapReduce, Hadoop, multi-round algorithms, ma-trix multiplication, experiments, cloud I