Assessment of the autism spectrum disorder based on machine learning and social visual attention: a systematic review

[EN] The assessment of autism spectrum disorder (ASD) is based on semi-structured procedures addressed to children and caregivers. Such methods rely on the evaluation of behavioural symptoms rather than on the objective evaluation of psychophysiological underpinnings. Advances in research provided evidence of modern procedures for the early assessment of ASD, involving both machine learning (ML) techniques and biomarkers, as eye movements (EM) towards social stimuli. This systematic review provides a comprehensive discussion of 11 papers regarding the early assessment of ASD based on ML techniques and children¿s social visual attention (SVA). Evidences suggest ML as a relevant technique for the early assessment of ASD, which might represent a valid biomarker-based procedure to objectively make diagnosis. Limitations and future directions are discussed.The authors have no relevant financial or non-financial interests to disclose. This work was supported by the Spanish Ministry of Economy, Industry, and Competitiveness funded project "Immersive Virtual Environment for the Evaluation and Training of Children with Autism Spectrum Disorder: T Room" (IDI-20170912). This work was also supported by the Spanish Ministry of Science and Innovation funded project "T-EYE: Monitoring system for children with ASD based on artificial intelligence and physiological measures" (IDI-20201146)Minissi, ME.; Chicchi-Giglioli, IA.; Mantovani, F.; Alcañiz Raya, ML. (2021). Assessment of the autism spectrum disorder based on machine learning and social visual attention: a systematic review. Journal of Autism and Developmental Disorders. 1-16. https://doi.org/10.1007/s10803-021-05106-5116Alcañiz Raya, M., Chicchi Giglioli, I. A., Marín-Morales, J., Higuera-Trujillo, J. L., Olmos, E., Minissi, M. E., Teruel Garcia, G., Sirera, M., & Abad, L. (2020). Application of supervised machine learning for behavioral biomarkers of autism spectrum disorder based on electrodermal activity and virtual reality. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2020.00090Alcañiz Raya, M., Giglioli, I. A. C., Sirera, M., Minissi, E., & Abad, L. (2020). Biomarcadores del trastorno del especto autista basados en bioseñales, realidad virtual e inteligencia artificial. Medicina (Buenos Aires), 80(supl II), 31–36.Alcañiz Raya, M., Marín-Morales, J., Minissi, M. E., Teruel Garcia, G., Abad, L., & Chicchi Giglioli, I. A. (2020). Machine learning and virtual reality on body movements’ behaviors to classify children with autism spectrum disorder. Journal of Clinical Medicine, 9(5), 1260.American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). . American Psychiatric Association.Bölte, S., Bartl-Pokorny, K. D., Jonsson, U., Berggren, S., Zhang, D., Kostrzewa, E., Falck-Ytter, T., Einspieler, C., Pokorny, F. B., Jones, E. J., Roeyers, H., Charman, T., & Marschik, P. B. (2016). How can clinicians detect and treat autism early? Methodological trends of technology use in research. Acta paediatrica, 105(2), 137–144.Carette, R., Cilia, F., Dequen, G., Bosche, J., Guerin, J. L., & Vandromme, L. (2017). Automatic autism spectrum disorder detection thanks to eye-tracking and neural network-based approach. International conference on IoT technologies for healthcare (pp. 75–81). Cham: Springer.Carette, R., Elbattah, M., Dequen, G., Guérin, J., Cilia, F., & Bosche, J. (2019). Learning to predict autism spectrum disorder based on the visual patterns of eye-tracking scanpaths. In HEALTHINF (pp. 103–112). Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217–227.Chevallier, C., Kohls, G., Troiani, V., Brodkin, E. S., & Schultz, R. T. (2012). The social motivation theory of autism. Trends in Cognitive Sciences, 16(4), 231–239.Chita-Tegmark, M. (2016). Social attention in ASD: A review and meta-analysis of eye-tracking studies. Research in Developmental Disabilities, 48, 79–93.Choueiri, R. N., & Zimmerman, A. W. (2017). New assessments and treatments in ASD. Current Treatment Options in Neurology, 19(2), 6.Chuba, H., Paul, R., Klin, A., & Volkmar, F. (2003, November). Assessing pragmatic skills in individuals with autism spectrum disorders. In Presentation at the National Convention of the American Speech-Language-Hearing Association, Chicago, IL.Chumerin, N., & Van Hulle, M. M. (2006). Comparison of two feature extraction methods based on maximization of mutual information. 2006 16th IEEE signal processing society workshop on machine learning for signal processing (pp. 343–348). IEEE.Cilia, F., Aubry, A., Bourdin, B., & Vandromme, L. (2019). Comment déterminer les zones d’intérêt visuelles sans a priori? Analyse des fixations d’enfants autistes en oculométrie. Revue De Neuropsychologie, 11(2), 144–150.Cilia, F., Aubry, A., Le Driant, B., Bourdin, B., & Vandromme, L. (2019). Visual exploration of dynamic or static joint attention bids in children with autism syndrome disorder. Frontiers in psychology. https://doi.org/10.3389/fpsyg.2019.02187Constantino, J. N., & Gruber, C. P. (2005). Social responsiveness scale (SRS). Los Angeles: Western Psychological Services.Crippa, A., Salvatore, C., Perego, P., Forti, S., Nobile, M., Molteni, M., & Castiglioni, I. (2015). Use of machine learning to identify children with autism and their motor abnormalities. Journal of Autism and Developmental Disorders, 45(7), 2146–2156.Currenti, S. A. (2010). Understanding and determining the etiology of autism. Cellular and Molecular Neurobiology, 30(2), 161–171.Dawson, G., Hill, D., Spencer, A., Galpert, L., & Watson, L. (1990). Affective exchanges between young autistic children and their mothers. Journal of Abnormal Child Psychology, 18, 335–345.Dawson, G., Toth, K., Abbott, R., Osterling, J., Munson, J., Estes, A., & Liaw, J. (2004). Early social attention impairments in autism: Social orienting, joint attention, and attention to distress. Developmental Psychology, 40(2), 271.Dawson, G., Webb, S. J., & McPartland, J. (2005). Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies. Developmental Neuropsychology, 27(3), 403–424.Deng, Y., Manjunath, B. S., Kenney, C., Moore, M. S., & Shin, H. (2001). An efficient color representation for image retrieval. IEEE Transactions on Image Processing, 10(1), 140–147.De Bildt, A., Sytema, S., Ketelaars, C., Kraijer, D., Mulder, E., Volkmar, F., & Minderaa, R. (2004). Interrelationship between autism diagnostic observation schedule-generic (ADOS-G), autism diagnostic interview-revised (ADI-R), and the diagnostic and statistical manual of mental disorders (DSM-IV-TR) classification in children and adolescents with mental retardation. Journal of Autism and Developmental Disorders, 34(2), 129–137.Duan, H., Zhai, G., Min, X., Che, Z., Fang, Y., Yang, X., Gutiérrez, J., & Callet, P. L. (2019, June). A dataset of eye movements for the children with autism spectrum disorder. In Proceedings of the 10th ACM Multimedia Systems Conference (pp. 255–260).Elbattah, M., Carette, R., Dequen, G., Guérin, J. L., & Cilia, F. (2019). Learning clusters in autism spectrum disorder: image-based clustering of eye-tracking scanpaths with deep autoencoder. 2019 41st Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 1417–1420). IEEE.Forscher, P. S., Lai, C. K., Axt, J. R., Ebersole, C. R., Herman, M., Devine, P. G., & Nosek, B. A. (2019). A meta-analysis of procedures to change implicit measures. Journal of Personality and Social Psychology 117(3), 522–559. https://doi.org/10.1037/pspa0000160.Franzen, M. D., & Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychological assessment. In R. J. Sbordone & C. J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 91–112). Gr Press/St Lucie Press Inc.Frazier, T. W., Strauss, M., Klingemier, E. W., Zetzer, E. E., Hardan, A. Y., Eng, C., & Youngstrom, E. A. (2017). A meta-analysis of gaze differences to social and nonsocial information between individuals with and without autism. Journal of the American Academy of Child & Adolescent Psychiatry, 56(7), 546–555.Ghaziuddin, M., & Gerstein, L. (1996). Pedantic speaking style differentiates asperger syndrome from high-functioning autism. Journal of Autism and Developmental Disorders, 26(6), 585–595.Goldberg, J. H., & Helfman, J. I. (2010). Visual scanpath representation. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (pp. 203–210).Goldstein, S., & Ozonoff, S. (Eds.). (2018). Assessment of autism spectrum disorder. Guilford Publications.Ismail, M. M., Keynton, R. S., Mostapha, M. M., ElTanboly, A. H., Casanova, M. F., Gimel’farb, G. L., & El-Baz, A. (2016). Studying autism spectrum disorder with structural and diffusion magnetic resonance imaging: A survey. Frontiers in Human Neuroscience, 10, 211.He, Y., Su, Q., Wang, L., He, W., Tan, C., Zhang, H., Ng, M. L., Yan, N., & Chen, Y. (2019). The characteristics of intelligence profile and eye gaze in facial emotion recognition in mild and moderate preschoolers with autism spectrum disorder. Frontiers in psychiatry, 10, 402.Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.Hyde, K. K., Novack, M. N., LaHaye, N., Parlett-Pelleriti, C., Anden, R., Dixon, D. R., & Linstead, E. (2019). Applications of supervised machine learning in autism spectrum disorder research: a review. Review Journal of Autism and Developmental Disorders, 6(2), 128–146.Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448–456). PMLR. Jiang, M., & Zhao, Q. (2017). Learning visual attention to identify people with autism spectrum disorder. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3267–3276).Kamp-Becker, I., Albertowski, K., Becker, J., Ghahreman, M., Langmann, A., Mingebach, T., Poustka, L., Weber, L., Schmidt, H., Smidt, J., Stehr, T., Roessner, V., Kucharczyk, K., Wolff, N., & Stroth, S. (2018). Diagnostic accuracy of the ADOS and ADOS-2 in clinical practice. European Child & Adolescent Psychiatry, 27(9), 1193–1207.Kang, J., Han, X., Song, J., Niu, Z., & Li, X. (2020). The identification of children with autism spectrum disorder by SVM approach on EEG and eye-tracking data. Computers in Biology and Medicine, 120, 103722. https://doi.org/10.1016/j.compbiomed.2020.103722Kasari, C., Sigman, M., & Yirmiya, N. (1993). Focused and social attention of autistic children in interactions with familiar and unfamiliar adults: A comparison of autistic, mentally retarded, and normal children. Development and Psychopathology, 5, 403–414.Klin, A. (2018). Biomarkers in autism spectrum disorder: challenges, advances, and the need for biomarkers of relevance to public health. Focus, 16(2), 135–142.Klin, A., & Mercadante, M. T. (2006). Autism and the pervasive developmental disorders. Revista Brasileira De Psiquiatria, 28(Suppl. 1), s1–s2. https://doi.org/10.1590/S1516-44462006000500001Koirala, A., Yu, Z., Schiltz, H., Van Hecke, A., Koth, K. A., & Zheng, Z. (2019, June). An exploration of using virtual reality to assess the sensory abnormalities in children with autism spectrum disorder. In Proceedings of the 18th ACM International Conference on Interaction Design and Children (pp. 293–300).Le Couteur, A., Haden, G., Hammal, D., & McConachie, H. (2008). Diagnosing autism spectrum disorders in pre-school children using two standardised assessment instruments: the ADI-R and the ADOS. Journal of Autism and Developmental Disorders, 38(2), 362–372.Li, J., Zhong, Y., Han, J., Ouyang, G., Li, X., & Liu, H. (2020). Classifying ASD children with LSTM based on raw videos. Neurocomputing, 390, 226–238.Li, J., Zhong, Y., & Ouyang, G. (2018). Identification of ASD children based on video data. 2018 24th International conference on pattern recognition (ICPR) (pp. 367–372). IEEE.Lieberman, M. D. (2010). Social cognitive neuroscience.Liu, W., Li, M., & Yi, L. (2016). Identifying children with autism spectrum disorder based on their face processing abnormality: A machine learning framework. Autism Research, 9(8), 888–898.Liu, W., Yu, X., Raj, B., Yi, L., Zou, X., & Li, M. (2015). Efficient autism spectrum disorder prediction with eye movement: A machine learning framework. 2015 International conference on affective computing and intelligent interaction (ACII) (pp. 649–655). IEEE.Lord, C., Risi, S., DiLavore, P. S., Shulman, C., Thurm, A., & Pickles, A. (2006). Autism from 2 to 9 years of age. Archives of General Psychiatry, 63(6), 694–701.Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659–685. https://doi.org/10.1007/bf02172145Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. A. (1999). Diagnostic observation schedule-WPS (ADOS-WPS). Los Angeles: Western Psychological Services.Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2001). Autism diagnostic observation schedule. Los Angeles: Western Psychological Services.Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9(11), 2579–2605.Matlis, S., Boric, K., Chu, C. J., & Kramer, M. A. (2015). Robust disruptions in electroencephalogram cortical oscillations and large-scale functional networks in autism. BMC Neurology, 15(1), 97.Mello, R. F., & Ponti, M. A. (2018). Machine learning: A practical approach on the statistical learning theory. Springer.Mitchell, T. M. (1997). Machine learning.Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Linee guida per il reporting di revisioni sistematiche e meta-analisi: il PRISMA Statement. PLoS Med, 6(7), e1000097.Mundy, P., Sigman, M., Ungerer, J., & Sherman, T. (1986). Defining the social deficits of autism: The contribution of non-verbal communication measures. Journal of Child Psychology and Psychiatry, 27(5), 657–669.Naber, F. B., Bakermans-Kranenburg, M. J., van Ijzendoorn, M. H., Dietz, C., van Daalen, E., Swinkels, S. H., Buitelaar, J. K., & van Engeland, H. (2008). Joint attention development in toddlers with autism. European Child & Adolescent Psychiatry, 17(3), 143–152.Nguyen, G. H., Bouzerdoum, A., & Phung, S. L. (2009). Learning pattern classification tasks with imbalanced data sets. Pattern recognition, 193–208.Nosek, B. A., Hawkins, C. B., & Frazier, R. S. (2011). Implicit social cognition: From measures to mechanisms. Trends in cognitive sciences, 15(4), 152–159.Orrù, G., Monaro, M., Conversano, C., Gemignani, A., & Sartori, G. (2020). Machine learning in psychometrics and psychological research. Frontiers in Psychology, 10, 2970.Pan, J., Ferrer, C. C., McGuinness, K., O’Connor, N. E., Torres, J., Sayrol, E., & Giro-i-Nieto, X. (2017). Salgan: Visual saliency prediction with generative adversarial networks. ArXiv preprint arXiv1701.01081.Parsons, S. (2016). Authenticity in Virtual Reality for assessment and intervention in autism: A conceptual review. Educational Research Review, 19, 138–157.Parsons, T. D. (2016). Clinical neuropsychology and technology. Cham: Springer International Publishing.Paulhus, D. L. (1991). Measurement and control of response bias. Elsevier.Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238.Reaven, J. A., Hepburn, S. L., & Ross, R. G. (2008). Use of the ADOS and ADI-R in children with psychosis: Importance of clinical judgment. Clinical Child Psychology and Psychiatry, 13(1), 81–94.Rutter, M., Bailey, A., & Lord, C. (2003). SCQ. The Social Communication Questionnaire. Western Psychological Services.Shmueli, G. (2010). To explain or to predict? Statistical Science, 25, 289–310.Schopler, E., Reichler, R. J., DeVellis, R. F., & Daly, K. (1980). Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). Journal of Autism and Developmental Disorders. https://doi.org/10.1007/BF02408436Sterling, L., Dawson, G., Webb, S., Murias, M., Munson, J., Panagiotides, H., & Aylward, E. (2008). The role of face familiarity in eye tracking of faces by individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(9), 1666–1675.Strimbu, K., & Tavel, J. A. (2010). What are biomarkers? Current Opinion in HIV and AIDS, 5(6), 463.Swettenham, J., Baron-Cohen, S., Charman, T., Cox, A., Baird, G., Drew, A., et al. (1998). The frequency and distribution of spontaneous attention shifts between social and nonsocial stimuli in autistic, typically developing, and nonautistic developmentally delayed infants. Journal of Child Psychology and Psychiatry, 39, 747–753.Tager-Flusberg, H., Paul, R., & Lord, C. (2005). Language and communication in autism. Handbook of Autism and Pervasive Developmental Disorders, 1, 335–364.Tanaka, J. W., & Sung, A. (2016). The “eye avoidance” hypothesis of autism face processing. Journal of Autism and Developmental Disorders, 46(5), 1538–1552.Tao, Y., & Shyu, M. L. (2019). SP-ASDNet: CNN-LSTM based ASD classification model using observer scanpaths. 2019 IEEE International conference on multimedia & expo workshops (ICMEW) (pp. 641–646). IEEE.Thabtah, F. (2019). Machine learning in autistic spectrum disorder behavioral research: A review and ways forward. Informatics for Health and Social Care, 44(3), 278–297.Torii, I., Ohtani, K., & Ishii, N. (2016). Measurement of ocular movement abnormality in pursuit eye movement (PEM) of autism spectrum children with disability. 2016 4th Intl conf on applied computing and information technology/3rd intl conf on computational science/intelligence and applied informatics/1st intl conf on big data, cloud computing, data science & engineering (ACIT-CSII-BCD) (pp. 235–240). IEEE.Vu, T., Tran, H., Cho, K. W., Song, C., Lin, F., Chen, C. W., Hartley-McAndrew, M., Doody, K. R., & Xu, W. (2017). Effective and efficient visual stimuli design for quantitative autism screening: An exploratory study. 2017 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI) (pp. 297–300). IEEE.Wallace, S., Parsons, S., & Bailey, A. (2017). Self-reported sense of presence and responses to social stimuli by adolescents with ASD in a collaborative virtual reality environment. Journal of Intellectual & Developmental Disability, 42(2), 131–141.Wallace, S., Parsons, S., Westbury, A., White, K., White, K., & Bailey, A. (2010). Sense of presence and atypical social judgments in immersive virtual environments: Responses of adolescents with Autism Spectrum Disorders. Autism, 14(3), 199–213.Walsh, P., Elsabbagh, M., Bolton, P., & Singh, I. (2011). In search of biomarkers for autism: Scientific, social and ethical challenges. Nature Reviews Neuroscience, 12(10), 603–612.Wan, G., Kong, X., Sun, B., Yu, S., Tu, Y., Park, J., Lang, C., Koh, M., Wei, Z., Feng, Z., Lin, Y., & Kong, J. (2019). Applying eye tracking to identify autism spectrum disorder in children. Journal of Autism and Developmental Disorders, 49(1), 209–215.Wilkinson, K. M. (1998). Profiles of language and communication skills in autism. Mental Retardation and Developmental Disabilities Research Reviews, 4(2), 73–79.Wolfers, T., Floris, D. L., Dinga, R., van Rooij, D., Isakoglou, C., Kia, S. M., Zabihi, M., Llera, A., Chowdanayaka, R., Kumar, V. J., Peng, H., Laidi, C., Batalle, D., Dimitrova, R., Charman, T., Loth, E., Lai, M. C., Jones, E., Baumeister, S., … Beckmann, C. F. (2019). From pattern classification to stratification: towards conceptualizing the heterogeneity of Autism Spectrum Disorder. Neuroscience & Biobehavioral Reviews, 104, 240–254.World Health Organization [WHO]. (2019). Autism spectrum disorders. Available at: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders (Visited on April 1, 2021).Wu, D., José, J. V., Nurnberger, J. I., & Torres, E. B. (2018). A biomarker characterizing neurodevelopment with applications in autism. Scientific Reports, 8(1), 1–14.Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 1100–1122.Yi, L., Feng, C., Quinn, P. C., Ding, H., Li, J., Liu, Y., & Lee, K. (2014). Do individuals with and without autism spectrum disorder scan faces differently? A new multi-method look at an existing controversy. Autism Research, 7(1), 72–83