The Eye Movements and Scoring Stability of Test Evaluator Using Computer Moniters: An Example for Geography Assessment

[[abstract]]當試卷無法全面利用電腦自動評分時，另一種閱卷方式是將考生答案卷掃描為影像檔，閱卷者透過網路在電腦螢幕前閱卷。為瞭解閱卷者在電腦螢幕閱卷的眼球運動及評分的穩定定，本研究以眼動儀搜集閱卷者眼球運動的資料，以紙筆閱卷、電腦螢幕閱卷(無眼動儀)及電腦螢幕閱卷(戴眼動儀)三種閱卷工具搜集閱卷者的評分資料。實驗結果發現，在眼球運動方面，年齡較高且平日較少電腦之閱卷者的眨眼次數較多，且在更換試題之際眨眼次數明顯增多，眼動個數亦以年齡組最高者較多，不過各年齡閱卷者在各題的凝視時間都相當短促，而在換大題之際明顯增加；在評分穩定性方面，則發現受到答案形式及評分指標影響，代碼型答案的評分穩定性最高，專有名詞評分穩定性次之，作圖評分穩定性最低。[[abstract]]When a test cannot be scored automatically by the computer, one substitution is to scan the answer sheets and have evaluators read the image files through the computer network. In order to understand evaluators’ eye movements and scoring stability, the evaluators’ eye movements were monitored while scoring paper-and-pencil items, and while scoring computer items with or without an eye tracker. The results showed that the evaluators with less computer experience, and those among the older age groups exhibited more frequent eye blinking, especially while the items were being changed. The oldest age group had the highest eye movement activity. All three types of scoring showed very short gazing times on small items, and longer gazing time when shifting to larger items. Criteria for evaluation, and the format of answers played critical roles in determining scoring stability. Stability was highest for coding items, followed by items dealing with specialized terms, and the lowest stability when scoring mapping