Computer models solving intelligence test problems: Progress and implications

This is the author’s version of a work that was accepted for publication in Artificial Intelligence. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Artificial Intelligence 230 (2016) 74–107. DOI 10.1016/j.artint.2015.09.011.While some computational models of intelligence test problems were proposed throughout the second half of the XXth century, in the first years of the XXIst century we have seen an increasing number of computer systems being able to score well on particular intelligence test tasks. However, despite this increasing trend there has been no general account of all these works in terms of how they relate to each other and what their real achievements are. Also, there is poor understanding about what intelligence tests measure in machines, whether they are useful to evaluate AI systems, whether they are really challenging problems, and whether they are useful to understand (human) intelligence. In this paper, we provide some insight on these issues, in the form of nine specific questions, by giving a comprehensive account of about thirty computer models, from the 1960s to nowadays, and their relationships, focussing on the range of intelligence test tasks they address, the purpose of the models, how general or specialised these models are, the AI techniques they use in each case, their comparison with human performance, and their evaluation of item difficulty. As a conclusion, these tests and the computer models attempting them show that AI is still lacking general techniques to deal with a variety of problems at the same time. Nonetheless, a renewed attention on these problems and a more careful understanding of what intelligence tests offer for AI may help build new bridges between psychometrics, cognitive science, and AI; and may motivate new kinds of problem repositories. © 2015 Elsevier B.V. All rights reserved.This work has been partially supported by the EU (FEDER) and the Spanish MINECO under grants TIN 2010-21062-C02-02, TIN 2013-45732-C4-1-P and FPI-ME grant BES-2011-045099, and by Generalitat Valenciana PROMETEOII/2015/013. We thank the editor and reviewers for their thorough and insightful comments.Hernández Orallo, J.; Martínez Plumed, F.; Schmid, U.; Siebers, M.; Dowe, DL. (2016). Computer models solving intelligence test problems: Progress and implications. Artificial Intelligence. 230:74-107. doi:10.1016/j.artint.2015.09.011S7410723