Investigation of computer vision techniques for automatic detection of mild cognitive impairment in the elderly

There are huge amounts of elderly people who suffer from cognitive impairment worldwide. Cognitive impairment can be divided into different stages such as mild cognitive impairment (MCI) and severe cognitive impairment like dementia. Its early detection can be of great importance. However, it is challenging to detect the cognitive impairment in the early stage with high accuracy and low cost, when most of the symptoms may not be fully expressed.;Although there have been some big changes and progresses in the field of detecting and diagnosing the cognitive impairment in recent years, all these existing techniques have their own weaknesses. Regarding the weaknesses of the existing techniques, both the traditional face to face cognitive tests and computer-based cognitive tests have the problems with diagnosing the mild cognitive impairment. More specifically, some personal information like age, education and personality will influence the test results and need to be taken into consideration carefully.;While the neuroimaging techniques are widely used in clinics, their major weakness is the high expenses required in the screening stage. Besides, the neuroimaging techniques are often used to diagnose the cognitive impairment only when the patients are found to have serious cognitive problems.;As a result, there is a pressing need to find alternative methods to detect the cognitive impairment in the early stage with high accuracy and low cost. In fact, some research works suggest that automatic facial expression recognition is promising in mental health care systems, as facial expressions can reflect people's mental state. Whilst viewing videos, studies have shown that the facial expressions of people with cognitive impairment exhibit abnormal corrugator activities compared to those without cognitive impairment. As a result, analysis of the facial expressions has the potential to detect the cognitive impairment.;In this thesis, a novel strategy for cognitive impairment detection is proposed, which is significantly different from the traditional methods like cognitive tests and neuroimaging techniques. The proposed strategy takes advantages of visual stimuli in the experiment and it mainly uses facial expressions and responses to detect the cognitive impairment when the participants are presentenced with the visual stimuli. As a result, this novel strategy for cognitive impairment detection with acceptable accuracy and low cost is achieved.;I present a novel deep convolution network-based system to detect the cognitive impairment in the early stage and support mental state diagnosis and detection. In the system, there are three important units in the proposed cognitive impairment detection system including the interface to arouse the facial expression, the proposed facial expression recognition algorithm and the algorithm to detect the cognitive impairment through the evolution of emotions. Among the cognitive impairment detection system, facial expression analysis is an important part. For facial expression analysis, this research presents a new solution in which the deep features are extracted from the Fully Connected Layer 6 of the AlexNet, with a standard Linear Discriminant Analysis Classifier exploited to train these deep features more efficiently.;The proposed algorithms are tested in 5 benchmarking databases: databases with limited images such as JAFFE, KDEF and CK+, and databases with images 'in the wild' such as FER2013 and AffectNet. Compared with the traditional methods and state-of-the-art methods proposed by other researchers, the algorithms have overall higher facial expression recognition accuracy. Also, in comparison to the state-of-the-art deep learning algorithms such as VGG16, GoogleNet, ResNet and AlexNet, the proposed method also has good recognition accuracy, much shorter operating time and lower device requirements.;In order to verify the system, I first made an experiment design. Then, clinical experiments were carried out in Shanghai, under the support from Dr Xia Li who is the chief physician in Mental Health Centre in Shanghai. After the recruitment procedure, a group of elderly people including cognitively impaired people and cognitively healthy people were invited to take part in the experiments. After the experiments in Shanghai, I classified the experiment data and used the proposed system to process the data.;I compared the major differences in the emotion evolution, including angry, happy, neutral and sad, between the cognitively impaired people and cognitively healthy people when they were watching the same video stimuli. In the selected testing group, the system had an overall cognitive impairment recognition accuracy of 66.7% using KNN classifier based on their evolutions of emotions.There are huge amounts of elderly people who suffer from cognitive impairment worldwide. Cognitive impairment can be divided into different stages such as mild cognitive impairment (MCI) and severe cognitive impairment like dementia. Its early detection can be of great importance. However, it is challenging to detect the cognitive impairment in the early stage with high accuracy and low cost, when most of the symptoms may not be fully expressed.;Although there have been some big changes and progresses in the field of detecting and diagnosing the cognitive impairment in recent years, all these existing techniques have their own weaknesses. Regarding the weaknesses of the existing techniques, both the traditional face to face cognitive tests and computer-based cognitive tests have the problems with diagnosing the mild cognitive impairment. More specifically, some personal information like age, education and personality will influence the test results and need to be taken into consideration carefully.;While the neuroimaging techniques are widely used in clinics, their major weakness is the high expenses required in the screening stage. Besides, the neuroimaging techniques are often used to diagnose the cognitive impairment only when the patients are found to have serious cognitive problems.;As a result, there is a pressing need to find alternative methods to detect the cognitive impairment in the early stage with high accuracy and low cost. In fact, some research works suggest that automatic facial expression recognition is promising in mental health care systems, as facial expressions can reflect people's mental state. Whilst viewing videos, studies have shown that the facial expressions of people with cognitive impairment exhibit abnormal corrugator activities compared to those without cognitive impairment. As a result, analysis of the facial expressions has the potential to detect the cognitive impairment.;In this thesis, a novel strategy for cognitive impairment detection is proposed, which is significantly different from the traditional methods like cognitive tests and neuroimaging techniques. The proposed strategy takes advantages of visual stimuli in the experiment and it mainly uses facial expressions and responses to detect the cognitive impairment when the participants are presentenced with the visual stimuli. As a result, this novel strategy for cognitive impairment detection with acceptable accuracy and low cost is achieved.;I present a novel deep convolution network-based system to detect the cognitive impairment in the early stage and support mental state diagnosis and detection. In the system, there are three important units in the proposed cognitive impairment detection system including the interface to arouse the facial expression, the proposed facial expression recognition algorithm and the algorithm to detect the cognitive impairment through the evolution of emotions. Among the cognitive impairment detection system, facial expression analysis is an important part. For facial expression analysis, this research presents a new solution in which the deep features are extracted from the Fully Connected Layer 6 of the AlexNet, with a standard Linear Discriminant Analysis Classifier exploited to train these deep features more efficiently.;The proposed algorithms are tested in 5 benchmarking databases: databases with limited images such as JAFFE, KDEF and CK+, and databases with images 'in the wild' such as FER2013 and AffectNet. Compared with the traditional methods and state-of-the-art methods proposed by other researchers, the algorithms have overall higher facial expression recognition accuracy. Also, in comparison to the state-of-the-art deep learning algorithms such as VGG16, GoogleNet, ResNet and AlexNet, the proposed method also has good recognition accuracy, much shorter operating time and lower device requirements.;In order to verify the system, I first made an experiment design. Then, clinical experiments were carried out in Shanghai, under the support from Dr Xia Li who is the chief physician in Mental Health Centre in Shanghai. After the recruitment procedure, a group of elderly people including cognitively impaired people and cognitively healthy people were invited to take part in the experiments. After the experiments in Shanghai, I classified the experiment data and used the proposed system to process the data.;I compared the major differences in the emotion evolution, including angry, happy, neutral and sad, between the cognitively impaired people and cognitively healthy people when they were watching the same video stimuli. In the selected testing group, the system had an overall cognitive impairment recognition accuracy of 66.7% using KNN classifier based on their evolutions of emotions