Empirical research for software architecture decision making: an analysis

Context: Despite past empirical research in software architecture decision making, we have not yet systematically studied how to perform such empirical research. Software architecture decision making involves humans, their behavioral issues and practice. As such, research on decision making needs to involve not only engineering but also social science research methods. Objective: This paper studies empirical research on software architecture decision making. We want to understand what research methods have been used to study human decision making in software architecture. Further, we want to provide guidance for future studies. Method: We analyzed research papers on software architecture decision making. We classified the papers according to different sub-dimensions of empirical research design like research logic, research purpose, research methodology and process. We introduce the study focus matrix and the research cycle to capture the focus and the goals of a software architecture decision making study. We identify gaps in current software architecture decision making research according to the classification and discuss open research issues inspired by social science research. Conclusion: We show the variety of research designs and identify gaps with respect to focus and goals. Few papers study decision making behavior in software architecture design. Also these researchers study mostly the process and much less the outcome and the factors influencing decision making. Furthermore, there is a lack of improvements for software architecture decision making and in particular insights into behavior have not led to new practices. The study focus matrix and the research cycle are two new instruments for researchers to position their research clearly. This paper provides a retrospective for the community and an entry point for new researchers to design empirical studies that embrace the human role in software architecture decision making