Mixed-Effects State Space Models for Analysis of Longitudinal Dynamic Systems

The rapid development of new biotechnologies allows us to deeply understand the biomedical dynamic systems in more details and at a cellular level. Many of the subject-specific biomedical systems can be described by a set of differential or dif-ference equations which are similar to engineering dynamic systems. In this paper, motivated by HIV dynamic studies, we propose a class of mixed-effects state space models based on the longitudinal feature of the dynamic systems. State space models with mixed-effects components are very flexible in modelling the serial correlation of within-subject observations and between-subject variations. The Bayesian approach and the maximum likelihood methods for standard mixed-effects models and state space models are modified and investigated for estimating unknown parameters in the proposed mixed-effects state space models. In the Bayesian approach the full condi-tional distributions are derived and the Gibbs sampler is constructed to explore the posterior distributions. For the maximum likelihood method we develop a Monte Carlo EM algorithm with a Gibbs sampler step to approximate the conditional expectations in the E-step. Simulation results indicate that the two methods perform wellx. We apply the mixed-effects state space model to a data set from an AIDS clinical trial to il-lustrate the proposed methodologies. The proposed models and methods may also have potential applications in other biomedical system analyses such as tumor dynamics in cancer research and genetic regulatory network modeling. Key words and phrases: State space models, mixed-effects models, parameter estimation