The effect of personality in sample selection for usability testing

The aim of this thesis is to determine whether selecting usability testing participants on the basis of their personality, as measured by the Myers-Briggs Type Indicator (MBTI) extraversion/introversion scale, can enhance the results obtained in usability testing in a web context. This thesis assesses whether extraverted subjects can uncover a more significant subset of problems and, in particular, a larger number of more severe problems, than introverts during usability testing. If this were the case, then the process of usability testing can become more efficient by using participants who are able to provide the highest quality feedback and are able to find the greatest number of, and the most severe problems leading to a decrease in the number of necessary participants and a reduction in project costs and a higher return on investment. Forty-three randomly selected candidates were given the MBTI test. Of these, twenty qualified as either extraverts or introverts and were, therefore, selected to take part in the study. Two sample groups were constituted, one comprising ten extraverts and the other ten introverts. Each of these participants was then asked to take part in a formal laboratory-based usability testing with an e-commerce website. The participants were required to complete a total of five tasks. Performance metrics from these tasks were observed and recorded, with the main focus being on the number and severity of usability problems found by each of the participants. The severity rating was assigned based on an average severity score taken from four evaluators. This was then combined with the frequency of problem occurrence to give an overall severity rating to the usability problem ranging from two to eight, with two representing a low-level cosmetic problem and eight representing a major usability problem, which rendered the system unusable. The study established that extraverts found more usability problems than introverts (p<0.001), and also a larger number of more severe problems. Additionally, extraverts found on average 96% of all unique usability problems, whereas introverts found just 28% of all such problems. A strong positive correlation was found between the degree of extraversion and the overall number of usability problems found, using a Pearson correlation coefficient R=0.85 (p<0.01). Extraverted participants were more confident in their feedback and more comfortable voicing their opinions than introverted participants. Overall, extraverts talked more often, as measured by the words-per-second metric (p