Experience from Measuring Program Comprehension—Toward a General Framework

Abstract: Program comprehension plays a crucial role during the software-develop-ment life cycle: Maintenance programmers spend most of their time with compre-hending source code, and maintenance is the main cost factor in software develop-ment. Thus, if we can improve program comprehension, we can save considerable amount of time and cost. To improve program comprehension, we have to measure it first. However, program comprehension is a complex, internal cognitive process that we cannot observe directly. Typically, we need to conduct controlled experiments to soundly measure program comprehension. However, empirical research is applied only reluctantly in software engineering. To close this gap, we set out to support researchers in planning and conducting experiments regarding program comprehen-sion. We report our experience with experiments that we conducted and present the resulting framework to support researchers in planning and conducting experiments. Additionally, we discuss the role of teaching for the empirical researchers of tomor-row.