Container-based Virtual Elastic Clusters

[EN] eScience demands large-scale computing clusters to support the efficient execution of resource-intensive scientific applications. Virtual Machines (VMs) have introduced the ability to provide customizable execution environments, at the expense of performance loss for applications. However, in recent years, containers have emerged as a light-weight virtualization technology compared to VMs. Indeed, the usage of containers for virtual clusters allows better performance for the applications and fast deployment of additional working nodes, for enhanced elasticity. This paper focuses on the deployment, configuration and management of Virtual Elastic computer Clusters (VEC) dedicated to process scientific workloads. The nodes of the scientific cluster are hosted in containers running on bare-metal machines. The opensource tool Elastic Cluster for Docker (EC4Docker) is introduced, integrated with Docker Swarm to create auto-scaled virtual computer clusters of containers across distributed deployments. We also discuss the benefits and limitations of this solution and analyse the performance of the developed tools under a real scenario by means of a scientific use case that demonstrates the feasibility of the proposed approach.This work has been developed under the support of the program "Ayudas para la contratacion de personal investigador en formacion de catheter predoctoral, programa VALi+d", grant number ACIF/2013/003, from the Conselleria d'Educacio of the Generalitat Valenciana. The authors wish to thank the financial support received form The Spanish Ministry of Economy and Competitiveness to develop the project "CLUVIEM", with reference TIN2013-44390-R.Alfonso Laguna, CD.; Calatrava Arroyo, A.; Moltó, G. (2017). Container-based Virtual Elastic Clusters. Journal of Systems and Software. 127:1-11. https://doi.org/10.1016/j.jss.2017.01.00711112