Nonparametric techniques with missing data

© 2021 Ruoxu TanThe thesis mainly studies three different topics on missing data, where we intend to develop nonparametric methods, to prove their theoretical properties and to apply them to simulated and real data. We begin with a brief introduction to missing data analysis in Chapter 1. The taxonomy of missing data mechanisms is presented, in which we emphasize importance and subtlety of missing at random (MAR) mechanisms. We then outline three popular techniques that are able to deal with missing data from a general point of view, namely, maximum likelihood estimation, multiple imputation and inverse probability weighting. Our first two topics of regression problems with missing data then follow. In Chapter 2, we consider nonparametric regression where both the covariate and the outcome variable are subject to missingness. Since MAR is not an appropriate assumption in such nonmonotone missing data context, we shall assume a missing not at random (MNAR) mechanism. Auxiliary variables are included to better model the missing data mechanism. We adapt a recently developed nonmonotone missing data model to our context and construct the corresponding inverse probability weighting local polynomial estimator. It is well known that if the weighting probability is too close to zero, i.e. the inverse probability weighting is too large, at some fully observed cases, the resulting estimator deteriorates. To overcome this issue, we introduce a constrained maximum likelihood estimation and an estimating equations method to ensure that the resulting weighting is bounded for fully observed cases. We prove asymptotic normality for the resulting regression estimator. Simulation results and a real data example show good practical performance of our method. In Chapter 3, we develop regression techniques for group testing data with missingness. When a new disease spreads quickly, time constraints and limited resources often prevent from testing each individual. For example, while covid-19 was spreading quickly worldwide in early 2020, many countries could only perform a restricted number of tests. The group testing procedure was introduced in the 1940's to handle such situations and estimate fast and accurately a low prevalence. Instead of testing all individuals for a disease, it tests pooled specimens of groups of individuals. Missing specimens are introduced in group testing data for the first time. We construct consistent nonparametric estimators of conditional prevalence for group testing data, designed for this case. We also consider the more difficult situation where the explanatory variable is also missing for some individuals. There we derive identifiability conditions and construct parametric likelihood estimators. We illustrate the numerical performance of our methods through simulations and real data examples. Extensions including imperfect tests, multivariate covariates are also included. Chapter 4 forms a connecting link to preceding chapters and Chapter 5. In this chapter, we briefly introduce functional data analysis. The famous functional principal component analysis is presented, followed by a review of selected topics, including missing data problems in the functional data context. Finally, in Chapter 5, our third topic considers registration and reconstruction for truncated functional data. Functional data usually possess structural complexities, e.g. amplitude and phase variations, as well as observational difficulties, e.g. missingness and measurement errors. We review and justify an existing curve registration method which was designed for complete data and generalize it to the case of truncated data. We then reconstruct the registered truncated curves and the corresponding warping functions, respectively. Finally, by warping the reconstructed curves back, it induces a new method of reconstructing truncated functional data taking phase variation into account. Numerical study shows good performance of our method. A real data example of animal growth is also presented