Predicting individual quality ratings of compressed images through deep CNNs-based artificial observers

Unlike traditional objective approaches aimed at MOS prediction, subjective experiments provide individual opinion scores that allow, for instance, to estimate the distribution of users' opinion scores. Unfortunately, the current literature is lacking objective quality assessment approaches that simulate the process of a subjective test. Therefore, this work focuses on modeling an individual subject through a deep CNN that, once trained, is expected to mimic the subject in terms of quality perception; for this reason, we call it "Artificial Intelligence -based Observer"(AIO). Several AIOs, modeling subjects with different characteristics, can be derived and used to simulate the process of a subjective test, thus yielding a more complete objective quality assessment. However, the training of the AIOs is hindered by two major issues: (i) the lack of training sets containing a large number of individual opinion scores; (ii) the noisy nature of individual opinion scores used as ground truth. To overcome these issues, we motivate a two-step learning approach. During the first learning step, the architecture of the well-known ResNet50 is appropriately modified and its initial weights are updated using a large scale synthetically annotated dataset of JPEG compressed images created for quality assessment purpose. This yields a new deep CNN called JPEGResNet50 that can accurately evaluate the quality of JPEG compressed images. The second learning step, conducted on a subjectively annotated dataset, refines the generic perceptual quality features already learned by the JPEGResNet50 to derive the AIO of each subject. Extensive computational experiments show the potential and effectiveness of our approach.Unlike traditional objective approaches aimed at MOS prediction, subjective experiments provide individual opinion scores that allow, for instance, to estimate the distribution of users' opinion scores. Unfortunately, the current literature is lacking objective quality assessment approaches that simulate the process of a subjective test. Therefore, this work focuses on modeling an individual subject through a deep CNN that, once trained, is expected to mimic the subject in terms of quality perception; for this reason, we call it "Artificial Intelligence -based Observer"(AIO). Several AIOs, modeling subjects with different characteristics, can be derived and used to simulate the process of a subjective test, thus yielding a more complete objective quality assessment. However, the training of the AIOs is hindered by two major issues: (i) the lack of training sets containing a large number of individual opinion scores; (ii) the noisy nature of individual opinion scores used as ground truth. To overcome these issues, we motivate a two-step learning approach. During the first learning step, the architecture of the well-known ResNet50 is appropriately modified and its initial weights are updated using a large scale synthetically annotated dataset of JPEG compressed images created for quality assessment purpose. This yields a new deep CNN called JPEGResNet50 that can accurately evaluate the quality of JPEG compressed images. The second learning step, conducted on a subjectively annotated dataset, refines the generic perceptual quality features already learned by the JPEGResNet50 to derive the AIO of each subject. Extensive computational experiments show the potential and effectiveness of our approach.