A Comparative Study of Three Multiple-Answer Test Formats

199 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 1987.Three multiple-answer test formats that may contain more than one correct answer were compared on mean test scores, reliability, and validity. While the Multiple True-False (MTF) test format requires the examinee's response of "T," or "F," or "DK" to each option of an item, the Multiple-Answer (MA) test format instructs the examinee to mark the correct options and leave the incorrect ones blank. If any examinee indicates omissions by blanks in an MA test, ambiguity would exist in interpreting each blank in the MA test.Two reading comprehension tests, one based on a narrative passage and the other an expository passage, were written in both MA and MTF formats. A sample of 334 eighth-grade students took either narrative or expository test in the MA-MTF order. The results suggested that omissions were indeed indicated as blanks in the MA tests, and this may affect their measurement characteristics. As expected, significant overestimation was found in MA test scores. Cronbach's alpha reliability estimates of the MTF tests were significantly higher than those of the MA tests. Using standardized reading comprehension scores as criterion, the highest validity coefficients in the total sample and all subgroups were yielded by MTF tests. However, the MTF test scores computed by 3 of 7 scoring formulas had significantly lower validity estimates than the MA scores. Superiority of MTF format to MA format is implied based on logical grounds and the evidences of higher MTF reliability and validity estimates.The Confidence Multiple True-False (CMTF) format requires the examinee to indicate the extent of confidence he/she has in each "T" and "F" decision. Information regarding the examinee's confidence facilitates a more detailed measurement of partial knowledge. Whether or not a better measure of the examinee's ability was obtained by the CMTF format was examined by the MTF and CMTF comparisons.Another sample of 155 eighth-grade students took the narrative or expository test in CMTF format. Although the CMTF alpha reliability coefficients and validity estimates were both higher than the MTF counterparts, the differences were nonsignificant. The practical use of the CMTF format was not warranted by an substantial improvement in reliability and validity estimates over the MTF format.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD