Latent variable models for characterizing the dynamic structure underlying complex behaviors

Latent variable models are a powerful tool for analyzing the dynamic structure underlying complex behavior in mammals. In this thesis, I discuss three applications of such mod- els and demonstrate their utility in uncovering novel scientific insights in mice engaging in decision-making and exploratory behaviors. In the first, we use a hidden Markov model with generalized linear model observations (GLM-HMM) to discover that mice occupy multiple internal states, or employ different strategies, when performing a binary decision-making task. This work also reveals that the contribution of the dorsomedial striatum (DMS) to decision-making behavior is state-dependent. In the second application, we extend the use of latent variable models to high-dimensional, continuous-time naturalistic behavior, using a switching Linear Dynamical System (sLDS) to identify discrete states in mice openly ex- ploring their environment. We find that this approach is superior to HMM-based methods in its ability to capture states that correspond to interpretable, consistent behavioral modules. Lastly, we detail novel approaches to improving inference in latent variable models, using ideas from spectral learning theory, with applications to both types of behavior. Overall, this work demonstrates the power of latent variable models to uncover hidden structure and insights into complex behavior in mice, which has broad implications for understanding neural circuits and behavior in both animal models and humans