Scaling the Evolution of Artificial Neural Networks in the Cloud

Deep Learning is one technique within the field of Machine Learning which is able to solve tasks such as Image Classification or Natural Language Processing. With its recent success, Deep Learning and therefore Artificial Neural Networks receive a high exposure. However, the design of architectures for Artificial Neural Networks still represents a time consuming and complex task. Therefore, automating the search for architectures of Artificial Neural Networks is the next step to simplify this process. The automation of this process should consider two aspects: The scalability to adapt to increasing problem sizes in a variety of domains and the performance of the resulting architectures. For this project, Neuroevolution was distributed in the Cloud using Kubernetes. Neuroevolution, a sub field of Evolutionary Computation, is able to evolve architectures of Artificial Neural Networks. The evolution is parallelisable, and therefore able to scale-out. Using cloud infrastructures with their theoretical endless resources, scalability and configuration of preferred hardware (CPUs, GPUs, or TPUs) is provided. Kubernetes abstracts the underlying resources, and enables a scalable and resilient execution of applications. Image classification is used for the evaluation of the implemented application. The experiments display that the application is able to scale horizontally with respect to strong and weak scaling. There is no work imbalance in the distribution evident, due to the asynchronous evolution. In addition, the average and maximum accuracy of evolved Artificial Neural Networks increases, and no overfitting is present in the obtained results. Due to the simplicity and scalability, Neuroevolution represents a viable solution for architecture search