Identifying Relevant Studies in Software Engineering

BACKGROUND: Systematic Literature Review (SLR) has become an important research methodology in software en-gineering since the introduction of Evidence-Based Software Engineering (EBSE) in 2004. One critical step in applying this methodology is to design and execute appropriate and effective search strategy. This is a time consuming and error-prone step, which needs to be carefully planned and implemented. There is an apparent need for a systematic approach to designing, executing, and evaluating a suitable search strategy for optimally retrieving the target literature from digital libraries. OBJECTIVE: The main objective of the research reported in this paper is to improve the search step of undertak-ing SLRs in SE by devising and evaluating systematic and practical approaches to identifying relevant studies in SE. OUTCOMES: We have systematically selected and analytically studied a large number of papers to understand the state-of-the-practice of search strategies in EBSE. Having identified the limitations of the current ad-hoc nature of search strategies used by SE researchers for SLRs, we have devised a systematic and evidence-based approach to devel-oping and executing optimal search strategies in SLRs. The proposed approach incorporates the concept of ‘Quasi-Gold Standard’, which consists of collection of known studies, and corresponding ‘Quasi-Sensitivity ’ into the search process for evaluating search performance. We report two case studies and their findings to demonstrate that the approach is able to improve the rigor of search process in an SLR, as well as it can serve as a supplement to the guidelines for SLRs in EBSE. We plan to further evaluate the proposed approach using a series of case studies on varying topics in software engineering. Key words: Search strategy, quasi-gold standard, systematic literature review, evidence-based software engineering 1