Prediction and machine learning: Explorations in psychological methodology

As the tremendous benefits of machine learning become clear, many scientific disciplines are currently experiencing the artificial intelligence revolution. Presently, these benefits have yet to be fully integrated into psychological methods. The goal of this PhD is to use this prediction framework to improve the methodology used by psychologists. We will utilize predictions and machine learning tools in two very different ways. In the first two parts we predict the outcome of computationally heavy problems, used to model affect time series, to avoid actually having to solve them. We are therefore able to greatly increase the speed of statistical inference for modelling emotional change. In the last three parts we use predictions to evaluate several state of the art psychological models and constructs which are often used in in current psychological research. By assessing their predictive quality we find that many of these new models and constructs have little incremental value over their much simpler predecessors. Specifically, first, we introduce fingerprint resampling, a generic method to speed up computationally very burdensome resampling methods such as bootstrapping and cross validation. In these resampling methods, a very similar function has to be optimized repeatedly. We propose to learn the relation between the resampled data sets and their corresponding optima. Using this learned knowledge we can predict future optima to drastically speed up the resampling procedure. In a second step, we introduce a similar solution to speed up the estimation of parameters for models without an analytical likelihood function. Here we propose a pooling of resources across researchers who have to do similar computations to estimate a certain model for their particular data set. If we are able structurally chart out the entire space of possible outcomes of a certain model, we can use machine learning techniques to predict, or solve, any future estimation problem on the spot, leading to a 100,000-fold speed up. Later we evaluate the lag-one vector autoregressive or VAR(1) model which is often used to model multivariate dynamical processes. Many parameters have to be estimated by this model which rises the questions whether it is not too complex. Using predictions we find that the VAR(1) model indeed overfits the data in a number of real data sets and that it is unable to outperform the much simpler AR(1) model. Last we focus on how we can extract meaningful information out of emotion dynamics which can predict the current psychological well-being of participants. Many different dynamic measures, such as autocorrelation, variance and density have all been claimed to evaluate a critical dynamical aspect of emotion time series, that can be linked to psychological well-being. We first show theoretically how many of these dynamical measures are severely confounded with the mean, especially when measurements are bounded. Finally we explore how all these measures are empirically related to each other and evaluate their incremental value in the prediction of psychological well-being. We find that, in many cases, the dynamical aspects of affective time series do not improve the prediction of well-being, above and beyond average levels of positive and negative affect.status: publishe