Software group reviews and the impact of procedural roles on defect detection performance

Software reviews (inspections) have received widespread attention for ensuring the quality of software, by finding and repairing defects in software products. A typical review process consists of two stages critical for defect detection: individual review followed by group review. This thesis addresses two attributes to improve our understanding of the task model: (1) the need for review meetings, and (2) the use of roles in meetings. The controversy of review meeting effectiveness has been consistently raised in the literature. Proponents maintain that the review meeting is the crux of the review process, resulting in group synergism and qualitative benefits (e.g. user satisfaction). Opponents argue that against meetings because the costs of organising and conducting them are high, and there is no net meeting gain. The persistence of these diverse views is the main motivation behind this thesis. Although commonly prescribed in meetings, roles have not yet been empirically validated. Three procedural roles (moderator, reader, recorder) were considered. A conceptual framework on software reviews was developed, from which main research questions were identified. Two experiments were conducted. Review performance was operationalised in terms of true defects and false positives. The review product was COBOL code. The results indicated that in terms of true defects, group reviews outperformed the average individual but not nominal group reviews (aggregate of individual reviews). However, groups have the ability to filter false positives from the individuals' findings. Roles provided limited benefits in improving group reviews. Their main function is to reduce process loss, by encouraging systematic consideration of the individuals' findings. When two or more reviewers find a defect during individual reviews, it is likely to be carried through to the meeting (plurality effect). Groups employing roles reported more 'new' false positives (not identified from preparation) than groups without roles. Overall, subjects' ability at the defect detection was low. This thesis suggests that reading technologies may be helpful for improving reviewer performance. The inclusion of an author role may also reduce the level of false positive detection. The results have implications on the design and support of the software review process