Experiments with Human Integration in Asynchronous and Sequential Multi-agent Frameworks for Architecture Optimization

AbstractArchitecting large complex systems is a challenging task due to the presence of uncertainty, ambiguity, and subjectivity as well as the extremely large space of candidate architectures. While the traditional approach to system architecting is a 100% human process, there has been a relatively recent trend to incorporate computational tools to different degrees in the process, thus making it more interactive. Tradespace exploration and optimization tools are among the most frequently used decision-support tools for systems architecting. From a mathematical perspective, architecture optimization problems are usually non-linear, non-convex, and multi-objective combinatorial optimization problems that are at least NP-hard. For these reasons, heuristics are often used to solve architecture optimization problems. These heuristics can be domain-specific, leveraging human knowledge or experience about the problem, or domain-independent, utilizing very little or no domain knowledge. Most applications of optimization in systems architecting consist of a single heuristic or meta-heuristic. However, this approach often results in suboptimal performance and premature convergence, in many cases due to the inability of a single heuristic to adapt to the changing environment of the search space. Humans, on the other hand, can also be considered as heuristics that may be able to adapt more easily to new situations and can naturally recognize patterns in information that can contribute to creating high performing architectures. A hyper-heuristic approach is proposed to combine multiple heuristics, including the human, and adapt the search strategy over time by applying heuristics that display good performance. In order to achieve a beneficial cooperation between humans and computers, this paper discusses and compares two different modes of a multi-agent optimization framework (asynchronous and sequential) that attempt to integrate human and computational agents in a hyper-heuristic architecture optimization algorithm. Experiments are conducted on a real-world problem to architect an Earth observing satellite system with a focus on gathering climate data