Replication package for the paper On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering

This repository contains the replication package for the paper "On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering", published in the 3rd IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), 2022. A preprint of the paper is included in this repository. Abstract—With the growing adoption of self-adaptive systems in various domains, there is an increasing need for strategies to assess their correct behavior. In particular self-healing systems, which aim to provide resilience and fault-tolerance, often deal with unanticipated failures in critical and highly dynamic environments. Their reactive and complex behavior makes it challenging to assess if these systems execute according to the desired goals. Recently, several studies have expressed concern about the lack of systematic evaluation methods for self-healing behavior. In the paper, we propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems that builds on chaos engineering. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. It has shown great promise in helping developers build resilient micro-service architectures and cyber-physical systems. CHESS turns this idea around by using chaos engineering to evaluate how well a self-healing system can withstand such perturbations. We investigate the viability of this approach through a case study that runs chaos experiments on a self-healing smart office environment. These experiments help us explore the promises and limitations of the approach, as well as identify directions where additional work is needed. We conclude with the lessons learned while evaluating self-healing systems at the infrastructural and functional level. Artifact The artifact provided here is a VMware image for a 64-bit Ubuntu 20.04LTS VM containing both the self-healing smart office environment (as a Kubernetes cluster) and the CHESS infrastructure used to conduct the self-healing and self-adaptation evaluations presented in the paper. There are detailed READMEs in the VM that discuss the experiments. Citation If you build on this artifact, please cite our work by referring to the paper: @inproceedings{naqvi2022:evaluating, title = {On Evaluating Self-Adaptive and Self-Healing Systems Using Chaos Engineering}, booktitle = {IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)}, author = {Naqvi, Moeen Ali and Malik, Sehrish and Astekin, Merve and Moonen, Leon}, year = {2022}, month = sep, publisher = {IEEE} keywords = {self-healing, resilience, chaos engineering, evaluation, exploratory study} }This work has been supported by the Research Council of Norway through the cureIT project (#300461). Sehrish Malik's work was in part supported through the ERCIM `Alain Bensoussan' Fellowship Programme