A study of the effects of user characteristics on mental models of information retrieval systems

grantor: University of TorontoThis study investigated effects of four user characteristics on users' mental models and search performance with information retrieval systems: educational and professional status, first language, academic background, and computer experience. Sixty-four subjects participated in the study. The repertory grid technique was used for eliciting and representing subjects' mental models. Using this method, nine concepts were chosen by four information retrieval experts to represent the components of information retrieval systems. Subjects were asked to rate these concepts on three attributes elicited from the same group of experts. Ratings were summarized using factor analysis techniques and the resulting factor patterns were analyzed to determine the representations of users' mental models. Subjects were asked as well to do four online searches. Search performance was measured by a relevance score and search time. The study found that educational and professional status and academic background had significant effects in differentiating users on both factor patterns and search performance measures. Computer experience had significant effects on factor patterns but not on search performance. First language had a borderline effect on factor patterns. But the effect was not significant enough at à = .05 level. It had no significant effect on search performance either. Librarians (professional) were significantly different from university students in their views of querying related concepts. Graduate students differed from undergraduate and high school students regarding the concepts related to data organization (such as data structure, document content, feedback, and so on) in information retrieval systems. Graduate and undergraduate students used significantly less time in searches than high school students. Science/engineering students differed significantly from social science/humanities students in their attitude towards browsing. The former group obtained higher relevance score than the second group. As for computer experience, subjects with little experience were more inclined to think the concepts related to querying part of information retrieval, such as query, search, and information need, as purposeful than those with greater experience. The experienced users were more likely to think the data organization related concepts applicable to all information systems than were the inexperienced ones. Subjects with a medium level of computer experience were more likely to consider browsing as purposeful than were those with low level experience. No differences in search performance were found when low, medium and high levels were directly compared. Cluster analyses were also performed on concept ratings. The findings were similar to those from the analyses on factor patterns. The findings suggest that when designing systems or developing user training programs, attention should be paid to the differences found between students and professionals, between educational levels, and between science and social science students. Skills of converting information needs into queries should be an emphasis in user training and components facilitating development of these skills should be incorporated into systems. Browsing navigating utilities may need to be enhanced for users with social sciences background. Users' computer experience should also be considered. Information retrieval systems should be designed to be compatible as much as possible to other computer applications.Ph.D