Quality adjusted Q-learning and conditional structural mean models for optimizing dynamic treatment regimes

The focus of this work is to investigate a form of Q-learning using estimating equations for quality adjusted survival time, and to generalize these methods to quality adjust other outcomes. We use the m-out-of-n bootstrap and threshold utility analysis to show how the patient-specific optimal regime varies according to treatment characteristics (e.g. cost, side effects). Methodologies investigated are demonstrated to construct optimal treatment regimes for the treatment of children's neuroblastoma. We also propose a new method for optimizing dynamic treatment regimes using conditional structural mean models. The inverse-probability-of-treatment weighted (IPTW) or g-computation estimator is used at each stage to estimate what we call the `preliminary' optimal treatment regime, given patient information up to the current stage and prior treatment assignment. Essentially this tailors the optimal treatment assignment at the current stage, and provides an optimal strategy for the remaining stages given the information currently available. We compare this method for optimizing a dynamic treatment regime to Q-learning. Additionally, we propose a two step prescriptive variable selection procedure that supports the tailored optimization of dynamic treatment regimes using conditional structural mean models by eliminating from consideration any suboptimal treatment regimes and sifting out the covariates that prescribe the optimal treatment regimes. The methods described herein are meant to advance the field of dynamic treatment regimes, a field that has a substantial impact on public health. The treatment policies that come from DTRs, whether determined for the population as a whole or tailored for specific subgroups, can be used to guide and shape health policies that will ultimately lead to greater public health and safety