Clustering-oriented Multiple Convolutional Neural Networks for Single Image Super-resolution

This is the author accepted manuscript. The final version is available from Springer Verlag via the DOI in this record.In contrast to the human visual system (HVS) that applies different processing schemes to visual information of different textural categories, most existing deep learning models for image super-resolution tend to exploit an indiscriminate scheme for processing one whole image. Inspired by the human cognitive mechanism, we propose a multiple convolutional neural network framework trained based on different textural clusters of image local patches. To this end, we commence by grouping patches into K clusters via K-means, which enables each cluster center to encode image priors of a certain texture category. We then train K convolutional neural networks for super-resolution based on the K clusters of patches separately, such that the multiple convolutional neural networks comprehensively capture the patch textural variability. Furthermore, each convolutional neural network characterizes one specific texture category and is used for restoring patches belonging to the cluster. In this way, the texture variation within a whole image is characterized by assigning local patches to their closest cluster centers, and the super-resolution of each local patch is conducted via the convolutional neural network trained by its cluster. Our proposed framework not only exploits the deep learning capability of convolutional neural networks but also adapts them to depict texture diversities for super-resolution. Experimental super-resolution evaluations on benchmark image datasets validate that our framework achieves state-of-the-art performance in terms of peak signal-to-noise ratio and structural similarity. Our multiple convolutional neural network framework provides an enhanced image super-resolution strategy over existing single-mode deep learning models.This study was funded by the National Natural Science Foundation of China (No. 61671481) and Qingdao Applied Fundamental Research (No. 16-5-1-11-jch), the Fundamental Research Funds for Central Universities, the EPSRC ESR project through ADR Funding Feasibility study on a fully deployable resilient flooding predicting, monitoring and response system and the 2016/2017 University of Exeter Outward Mobility Academic Fellowship