The thesis is divided into two main parts: i) Nonparametric statistics on high-dimensional and functional spaces, and ii) Nonparametric statistics on Riemannian manifolds. In this part, we will summarize the major contributions of the thesis. Nonparametric statistics on high-dimensional and functional spacesIn statistical learning, we introduce a new notion entitled: scalable Gaussian process classifier. The proposal is more general than the usual Gaussian process classifier for representing and classifying data lying on high-dimensional spaces. It is more advantageous for learning the hyper-parameters of the mapping (embedding) that maps initial data into a low-dimensional (feature) space and those of the Gaussian process classifier thr...