Television content analysis : agreement between expert and naive coders

Agreement between trained and untrained coders in assessing television content was investigated. A model integrating the different approaches to content analysis was proposed. The model contains three dimensions: audience coders versus expert coders, microanalysis versus macroanalysis, and quantitative versus qualitative analysis. The audience versus expert coders facet of that model was evaluated by having university students watch and assess the content of 24 television programs chosen from prime-time on the basis of their popularity. They were not trained in content analysis and did not know the questions about which they were asked until after viewing their program. Their evaluations were compared with similar evaluations given previously by trained (expert) coders. Each of the 24 programs was watched by 5 male and 5 female naive coders (total N=240). The groups were balanced for ethnicity and socioeconomic status. A statistic developed especially for this research was used to compare the naive and expert ratings on 22 selected variables. The results indicated that untrained and trained coders in general evaluated the programs similarly. Moreover, the questions on which the experts tended not to agree (that is, were unreliable) were generally the same ones on which the untrained coders did not agree, both amongst themselves and with the experts.Arts, Faculty ofPsychology, Department ofGraduat