Learning Individualized Treatment Rules with Sequential and Multi-Outcome Data

Learning optimal individualized treatment rules (ITRs) has become increasingly important in the modern era of precision medicine. Many statistical and machine learning methods for learning optimal ITRs have been developed in the literature. In this dissertation, we propose several approaches to solve some important problems regarding the data generating process and the learning algorithm for estimating ITRs. In the first project, we improve the outcome of interest in a clinical trial using a sequentially rule-adaptive design. Each entering patient will be allocated with a high probability to the current best treatment for this patient, which is estimated using the past data based on machine learning algorithm. We discuss the tradeoff between the training and test performance of the learnt ITR in the framework of contextual bandits. We also develop a tool that combines martingale with empirical process for sequentially generated data to tackle the theoretical problem with dependent data that cannot be solved by existing techniques for i.i.d. data. In the second project, we focus on the multi-stage stationary treatment policy (MSTP), which prescribes treatment assignment probabilities using the same decision function over stages. We estimate and conduct statistical inference for the parameters of the MSTP in high-dimensional settings. We propose to estimate the MSTP based on a penalized doubly robust estimator of the value function, and construct confidence intervals of the low-dimensional parameters that we are interested in using a one-step estimator. The proposed method allows for a slow convergence rate of the nuisance parameters in the model with a guarantee of the $1 / \sqrt{n}$ convergence rate of our interested parameters. In the third project, we estimate the ITR that maximizes the primary outcome and causes little harm to auxiliary outcomes in the meanwhile. We propose a fusion penalty to encourage ITRs based on the primary outcome and auxiliary outcomes to yield similar recommendations, and optimize a surrogate loss function for estimation. We derive the non-asymptotic properties for the proposed method and show that the agreement rate between the estimated ITRs for primary and auxiliary outcomes converges faster to the true rate compared to methods without using auxiliary outcomes.Doctor of Philosoph