ON MARKOV AND HIDDEN MARKOV MODELS WITH APPLICATIONS TO TRAJECTORIES

Markov and hidden Markov models (HMMs) provide a special angle to characterize trajectories using their state transition patterns. Distinct from Markov models, HMMs assume that an unobserved sequence governs the observed sequence and the Markovian property is imposed on the hidden chain rather than the observed one. In the first part of this dissertation, we develop a model for HMMs with exponential family distribution and extend it to incorporate covariates. We call it HMM-GLM, for which we propose a joint model selection method. The proposed selection criterion is tailored for HMM-GLM aiming at a more accurate approximation of the Kullback-Leibler divergence; we seek improvement of the widely-used Akaike information criterion. The second and the third parts of this dissertation are about clustering trajectories with HMMs and Markov mixture models. The research interests for HMM clustering are to develop a less computationally expensive and more interpretable algorithm for HMM sequence clustering problem, based on the emission and transition features of the chains. We propose an efficient clustering method using Bhattacharyya affinity to measure the pairwise similarity between sequences and apply a spectral clustering algorithm to obtain the cluster assignment. The computational efficiency benefits from the fact that our method avoids iterative computation for the affinity of a pair of sequences. Meanwhile, both simulation and empirical studies show that the proposed algorithm maintains good performance compared to other similar methods. In the third part of the dissertation, we address a study of the course of children and adolescents with bipolar disorder. Measuring and making sense of the fluctuations in different moods over time is challenging. We use a Markov mixture model with different transition matrices to find homogeneous clusters and capture different longitudinal mood change patterns. We also conduct a simulation study to investigate the performance of the model when there are violations of model assumptions. The results show that this model is fairly robust in the tested situations. We find that the clusters separate out those who tend to stay in a mood state from those who fluctuate between mood states more frequently