A Case for Replication: Synthesizing Research Methodologies in Software Engineering

Software Engineering (SE) problems are—from both prac-tical and theoretical standpoints—immensely complex, in-volving interactions between technical, behavioral, and so-cial forces. In an effort to dissect this complexity, SE re-searchers have incorporated a variety of research methods. Recently, the field has entered a paradigm shift—a broad awakening to the social aspects of software development. As a result, and in concert with an ongoing struggle to estab-lish SE research as an empirical discipline, SE researchers are increasingly appropriating methodologies from other fields. In the wake of this self-discovery, the field is entering a pe-riod of methodological flux, during which it must establish for itself effective research practices. We present a unifying framework for organizing research methods in SE. In the process of elucidating this framework, we dissect the cur-rent literature on replication methods and place replication appropriately within the framework. We also further clarify, from a high level and with respect to SE, the mechanisms through which science builds usable knowledge. 1