Everyday orchestration with Docker on Kubernetes

Many different industries and applications can benefit from the run-time management of containerised applications that is offered by a container orchestrator such as Kubernetes. Orchestrators can provide one-click deployment of a complex microservices architecture, smart scheduling across a cluster of compute nodes, self-healing abilities, automated scaling, zero-downtime rolling updates and much more. Science Gateways providing an interface for researchers looking to deploy applications in either the long- or short-term are an ideal candidate for this technology. However, deploying a Kubernetes cluster and writing Kubernetes manifests for each microservice are technically complex operations with a high learning curve. An initial investment of effort to prepare the cluster is equalled by the effort required for day-to-day cluster management and the constant need to author new manifests for new applications. Developers of Science Gateways may struggle with efficiently deploying and maintaining a backend Kubernetes cluster and likewise, users of Science Gateways cannot be expected to develop the native Kubernetes manifests required to deploy their applications. We propose a pair of tools aimed at empowering developers of Science Gateways with easier access to the benefits afforded by container orchestration: DocKubeADT and MiCADO. MiCADO is a one-click Kubernetes cluster deployed with Ansible. On top of the standard Kubernetes offering, MiCADO adds vendor-free auto-scaling for both containers and the underlying virtual machines, as well as improved security and monitoring. MiCADO offers a REST API for the submission, update and deletion of Application Description Templates (ADT), which define the size of the Kubernetes cluster and the applications to schedule across it. The ADT is based heavily on the generic TOSCA format proposed by OASIS. DocKubeADT is a new tool that translates a Docker-Compose file to MiCADO’s native ADT format. Docker container owners no longer need to learn an orchestrator specific format but can instead rely on the familiar toolchain they use when deploying their container locally. DocKubeADT compiles the Docker descriptor to a native Kubernetes manifest with Kompose1, and then further compiles it to an ADT, ensuring correctness. Properly placed in the submission pipeline, DocKubeADT can be used to submit a common Docker-Compose file directly to MiCADO, which in turn realises the Kubernetes deployment of all services described in the Docker-Compose, ready to benefit from the self-healing and smart scheduling of Kubernetes. DocKubeADT is already being used in the DIGITbrain platform, which empowers manufacturing SMEs with easy access to Digital Twins via a cloud-based marketplace. Future work will see DocKubeADT embedded in a Science Gateway for PITHIA-NRF, as part of an e-Science Centre for Space researchers