The competence of volunteer computing for MapReduce big data applications

Guo, W ORCiD: 0000-0002-3134-3327It is little to find off-the-shelf research results in the current literature about how competent Volunteer Computing (VC) performs big data applications. This paper explores whether VC scales for a large number of volunteers when they commit churn and how large VC needs to scale in order to achieve the same performance as that by High Performance Computing (HPC) or computing grid for a given big data problem. To achieve the goal, this paper proposes a unification model to support the construction of virtual big data problems, virtual HPC clusters, computing grids or VC overlays on the same platform. The model is able to compare the competence of those computing facilities in terms of speedup vs number of computing nodes or volunteers for solving a big data problem. The evaluation results have demonstrated that all the computing facilities scale for the big data problem, with a computing grid or a VC overlay being in need of more or much more computing nodes or volunteers to achieve the same speedup as that of a HPC cluster. This paper has confirmed that VC is competent for big data problems as long as a large number of volunteers is available from the Internet. © Springer Nature Singapore Pte Ltd. 2018