Variable Selection in Linear Models with Grouped Variables

Linear mixed models have been widely used for repeated measurements, longitudinal studies, or multilevel data. The selection of random effects in linear mixed models has received much attention recently in the literature. Random effects consider dependent structure between repeatedly measured data. Due to computational challenges, the selection of grouped random effects has yet to be studied. Grouped random effects, including genetics data or categorical variables, are commonly seen in practice. We present an efficient method for selecting random effects at group levels in linear mixed models. Specifically, the proposed method employs a restricted maximum likelihood function to estimate the covariance matrix of random effects. To achieve sparse estimation and grouped random effects selection, we then introduce a new shrinkage penalty term. In addition, we extend the idea of grouped variable selection onto the latent regression model. By incorporating regression onto latent traits, latent regression models provide a way to uncover hidden influential factors from the data and make more accurate predictions. Specifically, we develop a variable selection approach for latent regression item response theory models by introducing the group LASSO penalty into the marginal log-likelihood function of observed test responses. We derive the explicit forms of updating steps for model parameters in a modified Newton-Raphson method. Our approach selects significant covariates and estimates model parameters simultaneously. For both variable selection frameworks, we perform simulation studies to evaluate the variable selection performance of the proposed methods. We then compare them to existing or naive selection methods. Additionally, we apply the proposed methods on real data sets