Understanding object recognition performance at scale in machines and humans

This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 61-62).Machine performance on the object classication and detection tasks is remark- ably high today. On some datasets, such as ImageNet, it seems to surpass human performance according to recently published results. Yet when we run detectors over real videos we observe that machine performance is far inferior to human performance. We aim to resolve this disconnect and understand the true state of machine and human performance for object recognition. To do this we have gathered a new large image dataset, via the use of Amazon Mechanical Turk, with novel methodology and evaluation mechanisms to both answer questions about how well humans recognize objects and to carefully characterize machine performance. We have found that the performance of current state-of-the-art object detectors drops significantly when run on our dataset: from 71% accuracy to 25% accuracy accuracy. This drop in performance indicates that object detection is not a solved problem, despite previous benchmarks.by David Isaac Mayo.M. Eng.M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Scienc