Memory & Cognition (2018) 46:1315–1330https://doi.org/10.3758/s13421-018-0839-zMost evidence for the compensation account of cognitive trainingis unreliableTomasz Smolen´1 · Jan Jastrze¸bski1 · Eduardo Estrada2 · Adam Chuderski1Published online: 16 August 2018© The Author(s) 2018AbstractCognitive training and brain stimulation studies have suggested that human cognition, primarily working memory andattention control processes, can be enhanced. Some authors claim that gains (i.e., post-test minus pretest scores) from suchinterventions are unevenly distributed among people. The magnification account (expressed by the evangelical “who haswill more be given”) predicts that the largest gains will be shown by the most cognitively efficient people, who will also bemost effective in exploiting interventions. In contrast, the compensation account (“who has will less be given”) predicts thatsuch people already perform at ceiling, so interventions will yield the largest gains in the least cognitively efficient people.Evidence for this latter account comes from reported negative correlations between the pretest and the training/stimulationgain. In this paper, with the use of mathematical derivations and simulation methods, we show that such correlations are purestatistical artifacts caused by the widely known methodological error called “regression to the mean”. Unfortunately, moreadvanced methods, such as alternative measures, linear models, and control groups do not guarantee correct assessmentof the compensation effect either. The only correct method is to use direct modeling of correlations between latent truemeasures and gain. As to date no training/stimulation study has correctly used this method to provide evidence in favor ofthe compensation account, we must conclude that most (if not all) of the evidence should be considered inconclusive.Keywords Training · Stimulation · Regression to the mean · Compensation effectIntroductionIn social sciences and other disciplines dealing with livingorganisms (e.g., medicine, agriculture), researchers oftenstudy the effects of interventions. Specifically, in cognitiveand developmental psychology, recent years have broughta multitude of studies focused on the positive effectsof training cognitive abilities such as working memory,attention, and reasoning. Although there is still heateddebate on whether the far transfer of a trained ability,such as the increase in reasoning ability when workingmemory is trained, is possible (Klingberg, 2010; Jaeggi,Eduardo Estrada and Adam Chuderski contributed equally. Tomasz Smolen´tomasz.smolen@up.krakow.pl1 Jagiellonian University, ul. Grodzka 52, 31-044 Krakow,Poland2 University of California, Davis, 135 Young Hall,One Shields Avenue, Davis, CA, USABuschkuehl, Jonides, & Perrig, 2008) or not (Colomet al., 2013; Redick et al., 2013). There is little doubtthat in terms of near transfer, the existing cognitivetraining methods are effective (Klingberg, 2010; Morrison& Chain, 2011; Shipstead, Redick, & Engle, 2012). Morerecent reports from neuroscience have even suggested thepossibility of enhancing cognitive processing via non-invasive transcranial electrical stimulation with direct oralternating currents (e.g., Jausˇovec & Pahor, 2017; Pahor &Jausˇovec, 2014; Polanı´a, Nitsche, Korman, Batsikadze, &Paulus, 2012; Santarnecchi et al., 2015, 2016).Besides the sheer effectiveness of training/stimulation forcognitive performance, a growing number of studies haveinvestigated whether the training/stimulation gain (i.e., thedifference in score between the performance recorded aftertraining/stimulation [posttest] and the baseline performancebefore training/stimulation [pretest]) is distributed evenlyin the trained sample (e.g., Holmes & Gathercole, 2013;Loosli, Buschkuehl, Perrig, & Jaeggi, 2012), or whethersome people can be trained/stimulated more effectively thanothers. Two contrasting kinds of findings have been made1316 Mem Cogn (2018) 46:1315–1330with regard to the uneven distribution of such gains, andas a result two competing theories have been developed(see Karbach & Unger, 2014; Lo¨vde´n, Brehmer, Li, &Lindenberger, 2012).The magnification account (gisted by the evangelical“who has will more be given”) predicts that the mostcognitively efficient people at pretest will show thelargest gains. Regarding cognitive training, this propositionassumes that learning to perform better on a given task,including acquisition of new skills and strategies, requiressubstantial involvement of cognitive resources. The moreresources a person can invest in the training, the largerthe gain. However, such evidence is relatively scant, andpertains primarily to teaching more effective cognitivestrategies to deal with a task (e.g., Bjorklund & Douglas,1997; Brehmer, Li, Mu¨ller, von Oertzen, & Lindenberger,2007; Kliegl, Smith, & Baltes, 1990; Kramer & Willis,2002; Swanson, 2014, 2015; Verhaeghen & Marcoen,1996). Only one paper (Foster, Harrison, Draheim, Redick,& Engle, 2017) has suggested magnification effectspertaining to regular working memory training; it reportedlarger gains in people from the third than from the firsttercile of working memory capacity after 20 sessions ofeither complex span or running memory task training.In contrast to the magnification account, the compensa-tion account of cognitive training (“who has will less begiven”) predicts that the most cognitively efficient peopleat pretest already perform at ceiling and are not able toimprove (will display negligible gains). Therefore, trainingwill yield the largest gains in the least cognitively efficientpeople, who still have room for improvement, thus allowingthem to catch up.Probably the most widely discussed example of com-pensation account can be found in the field of intelligence(Lee et al., 2012, 2015; Baniqued et al., 2014). It has beenargued that training strategies have a greater impact onperformance when subjects’ baseline performance is low(Gopher, Weil, & Siegel, 1989; Espejo, Day, & Scott, 2005).Some other examples of compensation account come fromchildren’s learning (Schneider, 2012), selective attention(Feng & Spence, 2007), executive functions (Karbach &Kray, 2016), life span development (Baltes, 1987), and fromthe field of expertise in which training can reduce differ-ences between low- and high-aptitude experts (Bjorklund &Schneider, 1996). The compensation effect is also proposedas an explanation for improvements which are observed instrong decline in frontal lobe tasks (Raz, 2000).One form of compensation proposition is the disusehypothesis, which assumes that cognitive decline incognitive abilities (e.g., in old age) may be caused bysuboptimal use of available resources by people who havenever increased their cognitive reserve (e.g., at a youngerage). According to the disuse hypothesis, the decline can bereduced in groups with diminished abilities by optimizingthe use of resources. However, such optimization wouldhave a negligible effect in groups which already functionat a near optimal level (Gatz et al., 2001; Ihle, Oris, Fagot,Maggiori, & Kliegel, 2016; Kliegel, Zimprich, & Rott,2004; Sorenson, 1933).Although, there exist studies which use brain imaging toshow that cognitive training results in increased activation inregions that are less activated in a lower performing group(Hampstead, Stringer, Stilla, Giddens, & Sathian, 2012),the majority of evidence (e.g., Ball, Edwards, & Ross,2007; Chan, Wu, Liang, & Yan, 2015; Cox, 1994; Dahlin,2011; Gaultney, Bjorklund, & Goldstein, 1996; Karbach,Strobach, & Schubert, 2015; Kattenstroth, Kalisch, Holt,Tegenthoff, & Dinse, 2013; Willis & Nesselroade, 1990;Zinke, Zeintl, Eschen, Herzog, & Kliegel, 2012; Zinkeet al., 2014) for the compensation account comes fromnegative correlations of baseline performance, and gainsfrom training.For example, in a sample of 41 children aged 9 to 12years (Dahlin, 2011), negative correlations were observed(up to about −.5) between initial performance on the SpanBoard, Digit Span, Stroop, and Raven Colored Matricestests, and a gain in performance on these tasks after fiveweeks of intensive working memory training using theRoboMemo task. Chan et al. (2015) trained 13 younger and12 older adults for ten days on an adaptive n-back task.They pre- and post-tested them using spatial/verbal n-backtasks and a finger sequence learning task. The negativecorrelations observed between baseline performance on thelatter tasks and the respective increase in performance aftertraining was as much as r = −.81. Zinke et al. (2012)trained 20 older adults with five WM tasks over the courseof ten sessions and observed baseline—gain correlations asstrong as r = −.89. The other related studies cited abovereported at least moderate compensation effects.In psychology, it is barely possible to observe correlationstrengths exceeding .8 (the upper limit for the strengthof correlation is defined by the square root of product ofthe reliabilities of correlated tasks, and the reliability ofpsychological tests rarely exceeds .8). Thus, one has to beespecially suspicious of the aforementioned evidence for thecompensation account. In fact, the aim of the present paperis to demonstrate that calculating the correlations of pretestscores and gain, as is commonly adopted by proponentsof this account, is a hallmark example of a statisticalartifact called “regression to the mean”, dating FrancisGalton (1886). By mathematical derivation (Section 2, seealso Johns, 1981; Lord, 1956; Wall & Payne, 1973) andMem Cogn (2018) 46:1315–1330 1317numerical simulations (Section 2), we will show that thestrong negative correlations of the baseline performanceand gains from training are always present in the data,and are driven by statistical properties of noisy repeatedmeasurements. Thus, existing evidence is not able tosupport the compensation account, and the conclusionsprovided by virtually all studies in this vein are disputable.However, we note that some authors are aware ofthe problems pertaining to the pretest-gain correlationcalculations, and in order to validate the compensationeffect they applied one of the two methods whichshould yield more correct assessment of the magnitudeof this effect: either (a) an alternative variable for thegain calculation in order to avoid repeated measurements(Santarnecchi et al., 2016), (b) a formal model that includesor excludes the baseline performance × gain interaction(e.g., structural equation model, SEM, Guye, Simoni, & vonBastian, 2017; Lo¨vde´n et al., 2012), or (c) comparison ofcontrol and experimental group (e.g., Dahlin, 2011; Karbachet al., 2015; Zinke et al., 2012, 2014). Unfortunately, notall of these methods constitute an improvement comparedto the (naı¨ve) correlation of pretest and gain. In particular,using a control group is effectively of no use if the variablesinvolved in the analysis are confounded. Only correctapplication of the active control group in order to investigateindividual differences in training/stimulation effects maygive unbiased estimation of the compensation effect. Allof the aforementioned studies used the control group tocompare the influence of the pretest score on the gain(instead of posttest) between the control and experimentalgroups. As we will show, the pretest and gain are related in away that makes such an analysis faulty and only comparisonof the relationship between pretest and posttest in controland experimental groups may allow biased conclusions tobe avoided.Most of the examples of studies in which the correlationbetween pretest and gain was used come from the cognitivetraining domain so we will refer to this field in thisarticle whenever in need for an example of such a study.However on a methodological and statistical level this fieldis not specific in any way and we would like to underlinethat the problem described, solutions tested, and the finalconclusions refer to every domain of empirical sciencesin which the pretest—manipulation—posttest design isapplied and the hypothesis on the relationship betweenpretest and gain is tested.The remainder of this article has four parts. First, weshow analytically that simple correlation of observed pretestand gain cannot serve as estimation of correlation of truepretest and gain. Second, we take a closer look at thestrength of the observed correlation, depending on severalboundary conditions. Third, we discuss possible correctmethods of estimating true correlation of pretest and gain.Finally, we analyze sample data with both correct andincorrect methods in order to evaluate their accuracy.Analytical derivation of the persistentnegative correlation between pretestand gainIn this section, we analyze the mathematical relationbetween one variable (e.g., pretest score) and anothervariable that is the result of linear combination of the firstvariable with a third variable (e.g., the difference betweenpost-test and pretest scores). This kind of statistical modelis often used to detect a nonlinear relationship betweentwo variables. For example, in line with the compensationaccount, one can predict that people with lower cognitiveability level will benefit to a greater extent from cognitivetraining than people with a higher ability level (who alreadyperform optimally). One can then correlate the pretestability test score with the difference between post-test andpretest (i.e., gain) and interpret the negative correlation thatis usually observed in such a case as a direct confirmationof the compensation hypothesis. It will be demonstratedthat such a method may not be the best idea. Put simply,correlation statistics calculated in this way do not providereliable estimation of the true correlation between pretestand gain.In the present argument, we make only one assumption:that observed measures constitute the sums of some true,unobserved values and random independent noise. This isdefinitely a very weak assumption in light of the fact thatmost psychological studies do not directly tap the constructsmeasured, but rely on tools (e.g., tests, questionnaires, etc.)that show only imperfect reliability.Let us consider two observed variables, O1 and O2. Eachis the sum of some true (unobserved) value (V1 and V2,respectively) and random noise (ε1 and ε2, respectively).Of interest is the relationship between true value V1 andthe difference between true values V2 and V1. Since onecannot directly access these true values, the relationship inquestion must be estimated by using the observed values ofvariables (i.e. O1 and D = O2 − O1).Our argument can easily be generalized onto anylinear relation between variables O1 and O2 defined asabove (which obviously does not need to come frompretests and posttest), but for the sake of simplicity wehenceforth focus on a convenient (and commonly reportedin cognitive training studies) example relation pertaining tothe difference between the two variables (gain).1318 Mem Cogn (2018) 46:1315–1330The correlation of O1 and D is a biased estimator ofthe correlation of V1 and . Specifically, the difference Dbetween O2 and O1 equals + ε1 + ε2:D = O2 − O1= V2 + ε2 − (V1 + ε1)= V2 + ε2 − V1 + ε1= V2 − V1 + ε1 + ε2= + ε1 + ε2.Note that −(V1 + ε1) equals −V1 + ε1 because ε1 isindependent random noise. Adding such a noise to anyvariable has the same effect as subtracting it from thatvariable.Now, why is it a bad idea to estimate the correlation of V1and (true pretest and gain) by assessing the correlation ofV1 + ε1 and + ε1 + ε2 (observed pretest and gain)? ThePearson product-moment correlation coefficient rX,Y of Xand Y equals the covariance of X and Y (cov(X, Y )) scaledby the product of the standard deviations of these variables(σXσY ):rX,Y = cov(X, Y )σXσY. (1)As the scales are irrelevant here, the means of X and Y canbe fixed to 0 and the product of their standard deviations(SDs) can be fixed to 1. In which case the denominator alsoequals 1. Consequently, the strength of correlation betweenthe variables equals the covariance between them.The covariance between X and Y is an expected valueof the product of differences between each variable and itsmean:cov(X, Y ) = E ((X − μX)(Y − μY )) .Because the means of X and Y both equal 0 (due to ourchoice of scale), the formula can be simplified to:cov(X, Y ) = E(XY ).As, by assumption, both X and Y are the sums ofvariables (X = A + α and Y = B + β), the latter formulacan be rewritten in the following way:cov(X, Y ) = E ((A + α)(B + β)) ,and rearranged to:cov(X, Y ) = E(AB + Aβ + Bα + αβ),which, due to the feature of linearity of expected valueequals:cov(X, Y ) = E(AB) + E(Aβ) + E(Bα) + E(αβ). (2)Now, let us interpret both X and Y as observedmeasures, both A and B as the true unobserved valueof some psychological variable, and finally both αand β as independent random noise. In consequence,certain corollaries can be derived from all the previousassumptions. First, from the zero mean of both X and Y(because of our choice of scale), and from the zero mean ofboth α and β (the intrinsic feature of noise), there followsthe zero mean of both A and B. The latter fact implies thatthe expected value of the product of any pair of variables inthe formula (2) equals the covariance of these two variables,cov(X, Y )=cov(A,B)+cov(A, β)+cov(B, α)+cov(α, β).When two variables are independent then their covari-ance equals 0. Thus, both cov(A, β) and cov(B, α) yieldzero because the random noise (e.g., β) of one measure (e.g.,Y ) cannot be related to the true value of another variable(e.g., A). Consequently:cov(X, Y ) = cov(A,B) + cov(α, β),therefore, the covariance of X and Y equals the covarianceof A and B if and only if the covariance of α and β equals0. From the above, it follows that if α and β are dependent,then cov(X, Y ) constitutes a biased estimator of cov(A,B).Consequently, when one is computing the correlationbetween observed variable O1 and the difference D betweenO1 and another observed variable O2, in fact one iscomputing the correlation between V1 + ε1 and + ε1 +ε2, even though one is aiming to estimate the correlationbetween true value V1 and the difference between truevalues V2 and V1. Correlation strength computed in this waywill be different than the true strength of the relationshipin question because random terms ε1 and ε1 + ε2 areclearly not independent. As a result, simply because of thestatistical properties of the measures used, a researcher willlikely observe an incorrect correlation strength between agiven pretest score and a relative gain in a post-test score,compared to the true strength (if any) of the relationshipbetween the pretest score and the gain.Analysis of several speciﬁc cases of incorrectpretest-gain correlationThe strength of correlation between pretest and gain wasanalyzed in several specific cases in order to assessthe magnitude of difference between the true versus theobserved correlation of these variables.How large is the difference between trueand observed pretest-gain correlation?First, the relationship between pretest and gain wasexamined in a case in which there is no relation betweenpretest and posttest and their variance is equal.Let us remind the reader that gain (D) is the differencebetween posttest (O2) and pretest (O1). Variance (σ 2) ofMem Cogn (2018) 46:1315–1330 1319the difference between variables is the sum of variances ofthe variables, decreased by the covariance between themmultiplied by two (feature of covariance):σ 2D = σ 2O1 + σ 2O2 − 2 cov(O1,O2).Knowing σ 2D , one can compute the covariance betweenpretest and gain (feature of covariance):cov(O1,D) =σ 2O2− σ 2O1 − σ 2D2, (3)then, by putting the formula (3) into the formula (1) one cancompute the correlation between the variables. Accordingly,when the variances of both pretest and post-test are equaland there is zero correlation between them, the expectedvalue of correlation between pretest and gain equals −.71.What determines the size of the difference betweentrue versus observed pretest-gain correlation?As the correlation of two variables can be affected by achange in the variance of one variable, given that the othervariable’s variance and both variables’ means are fixed,next it was examined how the difference between trueversus observed correlation varied depending on the relativedisparity in variance between pretest and post-test. Figure 1shows a decreasing hyperbolic relationship between pretestvariance (varying from 0.14 to 7.14, while the post-testvariance equaled 1) and the correlation between pretest andgain. This very function implies that with an increasing−1.00−0.75−0.50−0.250 2 4 6σ1 σ2Fig. 1 Relation between the ratio of pretest standard deviation (σ1) topost-test standard deviation (σ2), and the observed correlation betweenpretest and gain (r , solid line). The dotted line marks point of equalstandard deviations of pretest and post-test indicating r = −.71range of pretest values, the size of the difference betweenthe true versus the observed pretest-gain correlation willincrease, approaching r = −1 (i.e., the perfect negativecorrelation, while the true correlation is null) when thepretest SD becomes at least five times larger than theposttest SD.What is the size of the difference between trueversus observed pretest-gain correlationwhen thereactually is a relationship between pretest and gain?Let us assume that post-test is a linear function of pretest,given by the formula V2 = βV1 + ζ (where β is changedue to intervention and ζ is random noise). When βequals 1, there is no relationship between the pretest andthe gain (the change due to intervention does not dependon base performance). For values larger than 1, thereis a positive relationship (predicted by the magnificationaccount). For values smaller than 1, there is negativerelationship (predicted by the compensation account).In order to examine example cases of the relationshipbetween pretest and gain, we analyzed how the correlationbetween these variables depends on the relationshipbetween pretest and post-test (β), the variance of predictionerror of pretest and post-test (ζ ), and the variance ofmeasurement error of pretest and post-test (ε, since O2 =V2 + ε).The correlation (ρ) between true pretest V1 and true gain can be obtained from the covariance of these variablesand their standard deviations according to formula (1). Next,the covariance can be computed from the formula (featureof covariance):cov(V1,) =σ 2V2− σ 2V1 − σ 22. (4)The true pretest variance (σ 2V1 ) depends on the choice ofscale and can be set to 1. The true post-test variance (σ 2V2 )equals β2σ 2V1 + σ 2ζ (which follows from the definition ofV2). The true gain variance (σ 2) equals σ2V1+ σ 2V2 − 2βσ 2V1(obvious proof of the latter claim lies beyond the scope ofthis article).Analogously, we can infer the correlation (r) betweenthe observed pretest (O1) and the observed gain (D). Theobserved pretest variance (σ 2O1 ) equals σ2V1+ σ 2ε , the truepretest variance (σ 2O2 ) equals σ2O2+ σ 2ε , and the observedgain variance (σ 2D) equals σ2O1+ σ 2O2 − 2βσ 2V1 .Figure 2 shows the relationship between the true andobserved correlation of pretest and gain, the regressionslope, and both types of error (ζ and ε). True correlation(ρ) was computed according to the formula (1), whereascovariance was provided on the basis of the formula (4).Analogously the observed correlation (r) was computed on1320 Mem Cogn (2018) 46:1315–1330σε2 = 0.5 σε2 = 2 σε2 = 4σζ 2≈0σζ 2=2σζ 2=8−5 −3 −1 1 3 5 −5 −3 −1 1 3 5 −5 −3 −1 1 3 5−1.0−0.50.00.51.0−1.0−0.50.00.51.0−1.0−0.50.00.51.0βCorrelation ρ rFig. 2 True correlations (ρ) between true values (V1 and ) andobserved correlations (r) of observed values (O1 and D), as a functionof V2 regression coefficient on V1 (β), the variance (σ 2) in predictionerror (ζ ) and the variance in measurement error (ε). β smaller than1 expresses compensation effect; β greater than to 1, magnificationeffect; β equal to 1, lack of either of the effects. When variance inprediction error is null the true correlations is perfectly negative (forcompensation) or perfectly positive (for magnification). The larger thevariance in prediction error the weaker the correlation (both true andobserved) for both these effects. When variance in measurement erroris null, r is equal to ρ for all β (not shown on the plot). The largerthe variance in measurement error the more underestimated are bothcompensation and magnification effectsthe basis of the formulas (1) and (3). It can be noticed thatthe higher the variance of measurement error, the larger thediscrepancy between true and observed correlation. Also,the higher the variance of prediction error, the smaller thestrength of both true and observed correlation for all valuesof β.Evaluation of alternative methodsof analysis of the relationship betweenpretest and gainApart from the naı¨ve correlation between pretest and gain,there are three quite straightforward statistical methodsthat can be used to examine the relationship between onevariable and another variable that is a linear function of theformer. In the remainder of the paper, each such methodwill be tested against artificially generated data (with andwithout the compensation effect).The mathematically simplest method is to computethe correlation in question between the gain and anotherindependent measure of the true pretest score (e.g.,Santarnecchi et al., 2016). The noise in both variableswill be unrelated, but the method requires additional data(another measure). The second method is to test therelative fit to data of regression models that include eitherunit relationship (slope coefficient) between pretest andpost-test (no relationship between pretest and gain), orrelationship of magnitude other than 1 (positive or negativerelationship between pretest and gain). The mathematicallymost complex method is to define the pretest, the gain,and their relationship with either a graphical or a structuralequation model (e.g., Lo¨vde´n et al., 2012).Each method requires only data from the experimental(i.e., either trained or stimulated) group. Additionally, aresearcher may introduce the control group, and thensimply compare the compensation effects in both groups.If the experimental group yields a significantly strongerpretest × gain negative (underadditive) interaction thanthe interaction that would (naturally) occur in the controlgroup, then the compensation effect due to interventionmay be validly argued for. Unfortunately, the control grouprequires planning ahead, which is a substantial change tothe study’s design (e.g., doubling the sample size, definingthe active control procedure, providing that the groups differonly in this procedure, etc.). The following analyses aim todetermine if the control group actually is or is not necessaryfor any valid inferences pertaining to the compensationeffect.Correlation with another measureThe first method is to simply use a different measure ofperformance in the post-test. Such an alternative measurewill not share the measurement error with the gain basedon the original variable (or vice versa), so the resultingobserved correlation will not be a biased estimation of thetrue correlation between pretest and gain.Using regressionAlternatively, and especially when no parallel scores areavailable, a linear regression model can be used. Thehypothesis that the gain is not related to pretest can beexpressed in another way: the coefficient of regressionof post-test over pretest equals 1. More specifically, thehypothesis that a higher pretest value will yield a highergain value is equivalent to the hypothesis that the slopeof the regression line of the post-test over the pretestis higher than 1. On the other hand, the hypothesis thata higher pretest value will yield a lower gain valueis equivalent to the hypothesis that the slope of theregression line of the post-test over the pretest is lowerthan 1.Mem Cogn (2018) 46:1315–1330 1321For example, if a hypothesis states that the higher pretestvalue (O1), the higher the gain value (O2 − O1), one canfit two linear models, M1: O2 = α + O1 + ε and M2:O2 = α + βO1 + ε (with possible restriction: β > 1),and can compare them with a model comparison tool (e.g.,ANOVA). If model M2 proves a significantly better fit todata than model M1, then the hypothesis about the positiverelation can be considered corroborated.It should be mentioned that actually one is not allowedto use linear regression when independent variables containnoise because in such a case linear regression’s assumptionof weak exogeneity is undermined (Chesher, 1991). But letus acknowledge the elephant in the room: this assumptionsis hardly ever true. Most psychological studies measureindependent variables with error, and in practice this doesnot preclude using linear regression and similar tools instatistical analysis.Using graphical and structural equationmodelsThe third solution requires using a more powerful analyticalframework that allows for the direct modelling of relation-ships between observed (manifest) and theoretical (latent)variables. Two tools commonly used for this purpose aregraphical models (Koller & Friedman, 2009) and structuralequation models (SEM, Kline, 2016). Each avoids the pit-falls resulting from the interaction of pretest noise and gainnoise because they allow the relationship between the true(noise free) variances of pretest and post-test to be mod-eled. Both graphical models and SEM can be defined inmany ways, so there is probably no established form ofmodel for the problem discussed here (for examples seeSection 2). Although graphical/structural equation modelsare more powerful than regression models, they usuallyincur the cost of using larger samples and multiple manifestmeasures.Analysis of the validity of compensationeﬀect detection in simulated dataA series of simulations was performed in order to establishwhich (if any) of the methods described above yields anacceptable estimate of the compensation effect (when theeffect is present in the data), or which can validly detectthe lack of the effect (when it is absent). 10,000 simulationswere run in order to achieve a level of accuracy higherthan 99%, taking into account the established variance ofthe parameters of interest, the effect size and α value of.05. (Burton, Altman, Royston, & Holder, 2006). We used5% trimmed means of parameters estimated in simulateddata because, unlike the untrimmed mean, it is robust toa moderate number of outliers. The measures of errorwere estimated in the same way as parameters of interest(mean value of measures in sampled datasets). This methodgave estimates identical to the ones based on measures ofstandard error calculated as the standard deviations of thesimulated parameters of interest (see Schafer & Graham,2002).Values of p were not computed directly as trimmedmeans of simulated samples because when p is close to zero(e.g., when there is significant or near significant test result),the distribution of this statistic tends to be visibly skewed.Such a skewed distribution mean is vulnerable to possibleoutliers which are results of sampling errors. Instead, wesampled statistics whose distributions were more stable(e.g., t or χ2) and computed p values based on trimmedmeans of these statistics and respective degrees of freedom.The sampling error, estimation of uncertainty, andsignificance of the effects depends on the sample size. So,in order to cover a wide range of possible expressions of thecompensation effect, we introduced several sample sizes inthe simulations. These sample sizes were chosen in orderto reflect the ones used in the studies which we referencedas examples of questionable support for the compensationeffect. We tested the validity of the statistical methods insmall (N = 28 similarly as used by Karbach et al., 2015),medium (N = 48 similarly as used by Chan et al., 2015),large (N = 80 similarly as used by Zinke et al., 2014), andhuge (N = 300 as a kind of best case scenario) samples.Note that due to the differences in the methodology andthe statistical methods used, the N values are merely basedon sample sizes used in the cited studies rather than mirrorthem.The first dataset contained a real compensation effect,i.e., the true unobserved gain was forced to negativelycorrelate with the true pretest value. In the second dataset,the false compensation effect could only have appeared asa result of regression to the mean as the true gain wasuncorrelated with the pretest. The third dataset used in theanalysis including a control group was composed of twosets of data in equal proportions. The first half (imitatingthe experimental group) was generated identically to firstdataset (which included covariance between pretest andgain) while the second half (imitating the control group)was generated identically to the second dataset (contain nocovariance between pretest and gain).Artiﬁcially generated datasetsA sample of N data points (28, 48, 80, or 300), each definedin two dimensions (pretest and gain), was drawn from the1322 Mem Cogn (2018) 46:1315–1330Table 1 Faulty estimation of relationships between pretest and gain, with simple correlationDataset 1 (real compensation) 2 (regression to the mean)r 95%CI p r 95%CI pN = 28 −.59 [−.79,−.29] < .001 *** −.33 [−.62, .04] .076N = 48 −.6 [−.75,−.38] < .001 *** −.33 [−.56,−.06] .018 *N = 80 −.6 [−.72,−.44] < .001 *** −.33 [−.51,−.12] .0026 *N = 300 −.6 [−.67,−.52] < .001 *** −.33 [−.43,−.22] < .001 ***bivariate normal distribution with the following covariancematrix[1 −0.4−0.4 0.5].The first vector, generated with a mean of 0 and a varianceof 1, represented the values of the true unobserved pretest(V1). The second vector, with a mean of 0.8 and a varianceof 0.5, reflected the values of the true unobserved gain ().The pretest and gain were negatively correlated at r = −.57.The mean of V1 (0) is an average pretest value; a mean of (0.8) is an average value of increase in post-test comparedto pretest.Next, the values of the true unobserved post-test werecomputed as the sum of the two variables (V2 = V1 + ).Finally, the observed values of both pretest (O1 = V1 + ε1)and posttest (O2 = V2 + ε2) were calculated, where ε1 andε2 were random noise drawn from the normal distribution(N = 1000, μ = 0, σ = 0.7). In a similar way, thealternative observed measures of the pretest (O ′1 = V1 + ε3)and post-test (O ′2 = V2 + ε4) were computed.The second dataset was generated in the same way as thefirst, with the sole difference that there was no correlationbetween the true pretest and the true gain. The covariancematrix defining this dataset was as follows[1 00 0.5],thus, there was no real compensation effect in the data.The third dataset consisted of two subsets of equalsizes generated identically to the first and second datasets.Each subset was respectively labeled as “experimental” or“control” group. The sizes of the entire dataset was the sameas sizes of the first and second datasets (28, 48, 80, or 300).Analysis of compensation effect detectionNaı¨ve correlation of pretest and gainThe correlation of pretest and gain was significant and negativein all cases except the second dataset (spurious compen-sation) for N = 28; this means the method incorrectlysignaled as significant a correlation that was actually null inthe cases of medium, large, and huge samples (see Table 1).The correlation in the first dataset (all sample sizes) did notdiffer significantly from the true correlation (−0.57) and theestimation was only slightly magnified.Correlation of gain with alternative pretest measureIn contrast to the naı¨ve correlation computed in the firststep, the correlation between the gain and the alternativemeasure of pretest performance correctly detected the lackof a significant relationship between these two variablesfor all sample sizes. However, the method did not detectthe current relationship in small and medium sample ofthe first dataset. In the large and huge samples, for whichthe correlation was correctly identified, its strength wasunderestimated (see Table 2).Table 2 Correlation of gain with alternative pretest measureDataset 1 (real compensation) 2 (regression to the mean)r 95%CI p r 95%CI pN = 28 −.26 [−.57, .11] .15 0 [−.36, .36] .998N = 48 −.27 [−.51, .01] .06 0 [−.28, .28] .993N = 80 −.27 [−.46,−.053] .015 * 0 [−.22, .22] > .999N = 300 −.27 [−.37,−.16] < .001 *** 0 [−.11, .11] > .999* p < .05, ** p < .01, *** p < .001Mem Cogn (2018) 46:1315–1330 1323Table 3 Estimation of pretest-posttest slope with linear regression modelDataset 1 (real compensation) 2 (regression to the mean)B 95%CI p B 95%CI pN = 28 0.4 [0.08, 0.72] .001 ** 0.67 [0.29, 1.05] .088N = 48 0.4 [0.17, 0.64] < .001 *** 0.67 [0.39, 0.95] .022 *N = 80 0.4 [0.22, 0.58] < .001 *** 0.67 [0.46, 0.88] .0028 **N = 300 0.4 [0.32, 0.49] < .001 *** 0.67 [0.57, 0.77] < .001 ***Note: Value of p is computed in reference to B = 1 as a null hypothesis* p < .05, ** p < .01, *** p < .001Linear regression modelThe model Posttest = α + βPretest + ε was fit to allthe generated datasets. The estimated Shapiro–Wilk testsresults revealed that the residuals were normally distributedin all models (the smallest W = .96, p = .5, observed forno compensation dataset, N = 28) and the models werehomoscedastic (as revealed by the estimations of Breusch—Pagan test results, the largest BP [1] = 0.82, p = .49,observed for real compensation dataset, N = 28).For the first dataset, in which the compensation effectwas present, the linear models provided slope parameterssignificantly smaller than one, correctly indicating theexistence of the effect. However, for the second dataset,in which the compensation effect was absent, the modelsprovided the correct result (the slope was not significantlydifferent from 1) only for the small sample and falselysignaled compensation (B significantly smaller than one)for the medium, large, and huge samples (see Table 3).Structural equation modelsTwo alternative structural equation models were fitted toeach dataset. The models were based on Lo¨vde´n et al.(2012), who tackled a similar problem of estimating thecorrelation between some baseline performance and asubsequent gain. The models belong to the latent differencescore class; this means that the difference (gain) betweenthe observed variables (post-test and pretest) is representedas a latent variable, and one of the observed variables (e.g.,post-test) is the sum of another observed variable (e.g.,pretest) and the latent gain (McArdle, 2009). This approachlets us directly model all parameters of the difference (i.e.,mean, variance, covariance of the pretest with the change,etc.). Also, in the latent difference score model we canexamine the statistical properties of the change withoutactually calculating the change scores. Latent differencescore modeling is also fit for our purposes because it restson assumptions similar to the ones made in this article (seeMcArdle & Hamagami, 2001).Both models consisted of four manifest variables whichwere scores of four observed measures (see Fig. 3), O1,O2, O ′1, and O ′2, the pretest and the post-test of theprimary, and the pretest and the post-test of the alternativemeasure, respectively. The residual terms of each pairof manifest measures (i.e., primary and alternative) wereallowed to correlate. Both pretest scores were loaded bythe latent variable representing the unobserved pretestperformance (V1), and both post-test scores were loaded by1V1 V2V1*Δ*Δε1 ε'1 ε2 ε'2ρπ π'σVσΔσ1 σ'1 σ2 σ'2μVμΔμm μmO1 O'1 O2 O'2Fig. 3 Structural equation models (based on Lo¨vde´n et al., 2012).Squares represent manifest variables (O1 is the score in the observedpretest, O ′1 is the score in alternative measures of the observedpretest, and O2 and O ′2 are scores in post-tests). Circles representlatent variables, V1 reflects unobserved pretest performance, V2reflects unobserved post-test performance, and indicates gain. Thetriangle represents a constant. Each solid arrow represents a nonzeroparameter. Each named arrow represents a free parameter. The modelsdiffer only in the ρ parameter (dashed arrow), which representsthe correlation between variance of baseline performance (V1∗) andvariance of gain (∗). In the first model ρ is fixed to 0, whereas in thesecond model this parameter is free1324 Mem Cogn (2018) 46:1315–1330the unobserved post-test latent variable (V2). The post-testvariable was the sum of the pretest variable and anotherlatent variable reflecting gain (). As both V1 and werelinked to the constant term and variance of V1∗, and ∗ wasfixed to 1, their regression loadings on V1 and equaledthe standard deviation of the variables and therefore thecovariance between V2∗ and ∗ equaled the correlationbetween them. Two additional variables for V1 (V1∗) and (∗, with variance fixed to 1 in both) are specified onlyto have a direct estimation of the correlation between them.If we correlated the variables directly, the models would bemathematically equivalent to the tested ones, but we wouldhave a covariance estimated instead of a correlation. Also,the mean of the alternative measure was fixed to the constantterm. The sole difference between the two models was thatthe first did not allow for the correlation between varianceterms (V2∗ and ∗), whereas the second did include such acorrelation (Fig. 3, dashed line).It must be stressed here that fitting SEM for most ofsample sizes used in this simulation is generally a veryprecarious idea and should be done only in cases whenit can be demonstrated that the small sample size doesnot influence the reliability of the results (Kline, 2016). Incase of described simulations, the relatively small samplesizes are not problematic because (a) the large number ofsimulation removes problem of sampling error, (b) artificialgenerating of the data provides that tested models are trueand (c) that all sources of error are known and controlled.We fitted the models with Lavaan (0.5) using unstan-dardized input and maximum likelihood estimation. In thefirst dataset (real compensation), only the first model (whichincluded pretest-gain correlation) achieved a satisfactory fitfor all sample sizes. The second model (which assumed nocorrelation of pretest and gain) failed to meet the criteria ofacceptability for any sample size. Additionally, the param-eter of the correlation between pretest and gain (ρ) in thefirst model was significantly negative for all sample sizes. Inthe second dataset (no real compensation effect), both mod-els achieved satisfactory fit. The measures of fit were verysimilar in both models, with the second model displayinga moderate advantage (χ2 and CFI were either inconclu-sive or slightly favored the first model; RMSEA was eitherinconclusive or slightly favored the second model; AICand BIC slightly favored the second model). More explicitevidence in favor of the second model was provided by esti-mations of ρ. For all sample sizes, the parameter did notdiffer significantly from zero; in fact, the estimated valueof the parameter was very close to zero. Full comparison ofthe fit measures is demonstrated in Table 7. Table 4 presentsbrief comparison based on the relative χ2. This measure isused to test the hypothesis about the increase in model’s fitas free parameters are added to the model. The model whichhas more free parameters will always have better fit than themodel with lower number of free parameters because it ismore complex/flexible than the latter one but the increasein fit does not always compensates the increase in the com-plexity. The relative χ2 can be used to compare nestedmodels fitted to the same dataset taking into account boththe models’ fit and complexity. The significant differencemeans that the more complex model is the better one.Using control groupThe linear regression model including the interactionbetween group and pretest was fitted to the third dataset,which contained two halves (first, including covariancebetween pretest and gain—experimental group; and second,which did not include such a correlation—control group).The model was defined as follows: Posttest = α+βGroup+γ Pretest + δGroup×pretest + ε. The estimated Shapiro—Wilk test results revealed that the residuals were normallydistributed in all models (the smallest W = .96, p = .5,observed for N = 28) and the models were homoscedastic(as revealed by the estimations of Breusch—Pagan testresults, the largest BP [1] = 6.37, p = .19, observed forN = 300).Table 4 Brief of the comparison of the goodness-of-fit measures for the two structural equation modelsDataset 1 (real compensation) 2 (regression to the mean)Relative χ2 p ρ 95%CI Relative χ2 p ρ 95%CIN = 28 4.05 .044 * −.58 [−1,−.074] 0.89 .34 .057 [−.72, .83]N = 48 6.33 .012 ** −.57 [−.9,−.22] 0.85 .36 .034 [−.52, .57]N = 80 10 .0016 ** −.57 [−.82,−.31] 1.04 .37 .018 [−.39, .42]N = 300 36.17 < .001 *** −.57 [−.69,−.44] 0.81 .37 0.0045 [−.19, .2]Note: DF for relative χ2 equals 1For complete comparison see Table 7 in Appendix* p < .05, ** p < .01, *** p < .001Mem Cogn (2018) 46:1315–1330 1325Table 5 Estimation of interaction parameter (group × pretest) in linearregression model (dataset 3)B 95%CI pN = 28 −0.27 [−1.02, 0.48] .45N = 48 −0.27 [−0.8, 0.27] .31N = 80 −0.27 [−0.67, 0.14] .19N = 300 −0.27 [−0.47,−0.067] .0088 **For estimation of all parameters in the model see Table 8 in Appendix* p < .05, ** p < .01, *** p < .001The resulting model parameters are presented in Table 8in Appendix. Comparison of estimations of the interactionparameter is presented in Table 5. The interaction inquestion was significantly different from zero only for hugesample size; this means that the linear model failed to detectthat the experimental group displayed a larger compensationeffect than the control group for all “realistic” samplesizes. This fact led to the invalid conclusion that in theexperimental group the compensation effect did not consistof anything more than the artifactual compensation effect(as in the control group).ConclusionsThree patterns can be observed among the results of theapplication of the analysed statistical methods (see Table 6).First, both correlation with alternative measure and analysisof interaction using the control group seem to be ableto correctly diagnose the presence and absence of thecompensation effect, provided they have enough power.In the analyzed datasets, the correlation with alternativemeasure failed to detect the existent compensation insmall and medium samples; however, the method almostindicated the effect in the medium sample (p = .06)and correctly detected the effect in the large and hugesamples. Similarly, analysis of interaction using the controlgroup revealed a negative interaction between group andpretest value only in huge sample, therefore the hypothesisabout a higher compensation effect in the experimentalgroup (both real and artifactual compensation) than inthe control group (sole artifactual compensation) was notconfirmed for all sample sizes based on real studies.However, the estimate of the interaction was invariablynegative for all sample sizes and it was the large standarderror that made the effect insignificant. Nevertheless, theerror systematically decreased with the increase of samplesize and ultimately the interaction reached significance forthe huge sample. Therefore, one can undoubtedly expectthat if sufficient power is provided (low measurement error,large sample, etc.) this method will correctly detect anexisting compensation effect.Second, both the naı¨ve correlation and the linear modelappeared to be quite unreliable. On one hand, naı¨vecorrelation provided the correct diagnosis for small andmedium samples and the linear model provided correctdiagnosis for the small sample, but it is clear that neithermethod falsely diagnosed the inexistent compensation effectfor the small (or medium) sample size simply due tothe large standard error of the estimation, which wasconsequence of the sample size. Thus, when using thesemethods, one finds oneself in an awkward situation wherethe higher the power of the test, the higher the chance ofobtaining the wrong outcome.Finally, structural equation models proved to be anaccurate tool for the task. For all sample sizes of thefirst dataset, SEM unequivocally indicated the first of themodels (which assumed correlation between pretest andgain) as better than the alternative one. Also, estimation ofthe correlation parameter was definitely negative. So, theconclusion about the existence of the compensation effectwas consistently true. In the second dataset, SEM indicatedthe second model (which assumed no correlation betweenpretest and gain) as better. However, the fit measures werefar less unequivocal than in the first dataset. Some of the fitmeasures indicated the first model as slightly better, whilesome of them were inconclusive; however, in the big picturethe second model achieved a slightly better fit for all sampleTable 6 Diagnosis of compensation effect by all tested statistical methodsDataset 1 (real compensation) 2 (regression to the mean) 3 (combined dataset)NC CWAM LM SEM NC CWAM LM SEM LMWIN = 28 detected rejected detected detected rejected rejected rejected rejected not discriminatedN = 48 detected rejected detected detected detected rejected detected rejected not discriminatedN = 80 detected detected detected detected detected rejected detected rejected not discriminatedN = 300 detected detected detected detected detected rejected detected rejected discriminatedNote: Correct outcomes are printed in bold. NC—Naı¨ve correlation, CWAM – Correlation with alternative measure, LM—Linear model,LMWI—Linear model with interaction1326 Mem Cogn (2018) 46:1315–1330sizes. Moreover, the value of the correlation parameter inthe first model was estimated as close to zero.Thus, the best solution to the problem of testingthe compensation account is to express directly the true(measurement error free) variables in the fitted model(either SEM or graphical model) and test the relationshipbetween them. Acceptable, albeit much less trustworthy,solutions are either to use an alternative measure ofpretest/posttest instead of the primary one in the correlationtest, or to include a control group in the study and tocompare the possible compensation effect between thecontrol and the experimental (trained/stimulated) group.These methods can be used with the restriction that only apositive outcome of these tests is credible because a negativeoutcome can show that there is insufficient test power, orthat the effect is nonexistent. Finally, neither a simple linearmodel test nor a naı¨ve correlation should ever be used to testthe compensation account!DiscussionThis study aimed to methodologically assess the existingand potential evidence in favor of the popular compensationaccount of cognitive training and neuronal stimulation; itpredicts the training effect size to be negatively relatedto the baseline performance tested before the training.However, most such evidence consists of negative Pearsoncorrelations of pretest score and training gain, the latterexpressed as the difference between post-test and pretest. Arelatively simple mathematic derivation demonstrated thatsuch an outcome occurs naturally when gain (treated as thedependent variable) is the linear function of the independentvariable (pretest); that is a specific case of a moregeneral statistical artifact called regression to the mean.This conclusion was supported by numerical simulations,showing a robust tendency towards reporting negativecorrelations even when the pretest and posttest scores are infact unrelated. As a result, one must conclude that most ofthe existing evidence in favor of the compensation accountis questionable (e.g., Chan et al., 2015; Cox, 1994; Dahlin,2011; Gaultney et al., 1996; Karbach et al., 2015; Zinkeet al., 2012, 2014).Furthermore, using numerical simulations we exam-ined if the four alternative methods (correlation with analternative pretest measure, simple linear regression model,linear regression model including control group, and struc-tural equation model) can validly evaluate the magnitude ofthe compensation effect when it is present in data, as wellas validly report its lack when it is absent. Both the linearregression model and naı¨ve correlation yielded false alarms,detecting compensation whether or not it was present indata. On the other hand, correlation with an alternativemeasure failed to detect a true compensation effect in asmall sample. Also, including a control group and exam-ining the interaction between group and pretest value didnot lead to correct discrimination between spurious andreal compensation. However, similarly to correlation withanother measure, the low power of the tests was probably thereason that the effect was missed. The only fully valid detec-tion of compensation was achieved by the use of an SEM,which diagnosed properly in both datasets and for all samplesizes.However, the present study should not be interpretedas an argument against the compensation account, as theaccount itself might be valid. Simply, the empirical statusof this hypothesis is still indeterminate as the validityof the methods used to corroborate it is doubtful. Withproper methods, the account may in principle be supportedby future data. Consequently, the merit of the presentstudy lies in stimulating methodologically valid researchon the individual differences in training and stimulationeffects. Knowledge on such differences is very importantbecause it informs who should be primarily targeted byincreasingly common but costly cognitive training programs(Roma´n et al., 2016; Schmiedek, Lo¨vde´n, & Lindenberger,2010) or transcranial stimulation (Jausˇovec & Pahor, 2017;Santarnecchi et al., 2015), which might help varioussubpopulations to improve performance. In fact, somestudies (e.g., Au et al., 2015; Santarnecchi et al., 2016) thatused methods beyond naı¨ve correlation indeed suggestedthat people whose performance is worse at baseline mayespecially benefit from such programs. However, the onlytwo studies that validly applied an SEM (Guye et al., 2017;Lo¨vde´n et al., 2012) provided results that support boththe compensation account and the magnification account;this suggests that both compensation and magnification canoccur, depending on the faculty trained and the proceduresapplied (see Borella, Carbone, Pastore, Beni, & Carretti,2017). More reliable future studies are definitely neededbefore any firmer conclusions can be drawn. The mostimportant take-home message from the present analysis isthat such studies need reliable statistical methods.Acknowledgments Tomasz Smolen´ was supported by grant2015/18/E/HS6/00152 received from National Science Centre, Poland,Adam Chuderski was supported by grant 2013/11/B/HS6/01234received from National Science Centre, Poland.Open Access This article is distributed under the terms of theCreative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricteduse, distribution, and reproduction in any medium, provided you giveappropriate credit to the original author(s) and the source, provide alink to the Creative Commons license, and indicate if changes weremade.Mem Cogn (2018) 46:1315–1330 1327AppendixTable 7 Comparison of the goodness-of-fit measures of the two structural equation models for the two datasetsDataset 1 (real compensation) Dataset 2 (regression to the mean)Model 1 Model 2 Relative fit Model 1 Model 2 Relative fitN = 28χ2[DF ] 1.9 [2] 6.25 [3] 4.05 [1] 1.9 [2] 3 [3] 0.89 [1]p(χ2) .39 .1 .044 * .39 .39 .34RMSEA[95%CI ] .057 [.0004, .338] .167 [.034, .4] — .057 [.0003, .338] .057 [.0004, .296] —CFI .985 .908 — .99 .986 —AIC 322 325 — 332 332 —BIC 338 339 — 348 346 —ρ[95%CI ] −.58 [−1,−.074] — — .057 [−.72, .83] — —p(ρ) .11 — — .5 — —N = 48χ2[DF ] 1.82 [2] 8.41 [3] 6.33 [1] 1.87 [2] 2.92 [3] 0.85 [1]p(χ2) .4 .04 * .012 * .39 .4 .36RMSEA[95%CI ] .04 [.0001, .255] .174 [.052, .334] — .042 [.0002, .257] .042 [.0002, .224] —CFI .992 .916 — .994 .992 —AIC 545 550 — 562 561 —BIC 567 570 — 585 582 —ρ[95%CI ] −.57 [−.9,−.22] — — .034 [−.52, .57] — —p(ρ) .04 * — — .51 — —N = 80χ2[DF ] 1.84 [2] 12.32 [3] 10 [1] 1.84 [2] 2.84 [3] 1.04 [1]p(χ2) .4 .007 ** .0016 ** .4 .42 .37RMSEA[95%CI ] .032 [.0001, .198] .185 [.079, .308] — .032 [.0001, .198] .031 [0, .17] —CFI .995 .917 — .997 .996 —AIC 901 909 — 929 928 —BIC 929 935 — 958 954 —ρ[95%CI ] −.57 [−.82,−.31] — — .018 [−.39, .42] — —p(ρ) .004 ** — — .51 — —N = 300χ2[DF ] 1.8 [2] 38.2 [3] 36.17 [1] 1.77 [2] 2.78 [3] 0.81 [1]p(χ2) .41 < .001 *** < .001 *** .41 .42 .37RMSEA[95%CI ] .016 [0, .101] .196 [.143, .254] — .016 [0, .101] .015 [0, .087] —CFI .999 .915 — .999 .999 —AIC 334 338 — 345 345 —BIC 339 342 — 349 349 —ρ[95%CI ] −.57 [−.69,−.44] — — .0045 [−.19, .2] — —p(ρ) < .001 *** — — .5 — —*p < .05, ** p < .01, *** p < .0011328 Mem Cogn (2018) 46:1315–1330Table 8 Estimation of linear regression models parameters (dataset 3)Predictor B 95%CI pN = 28(Intercept) 0.8 [0.19, 1.41] .01*Group −0.006 [−0.87, 0.85] .988Pretest 0.67 [0.15, 1.19] .01*Group × pretest −0.27 [−1.02, 0.48] .45N = 48(Intercept) 0.8 [0.36, 1.25] .001 **Group −0.0033 [−0.64, 0.63] .991Pretest 0.67 [0.3, 1.05] .001 **Group × pretest −0.27 [−0.8, 0.27] .31N = 80(Intercept) 0.8 [0.46, 1.13] < .001 ***Group 0.0042 [−0.48, 0.48] .99Pretest 0.67 [0.38, 0.95] < .001 ***Group × pretest −0.27 [−0.67, 0.14] .19N = 300(Intercept) 0.8 [0.63, 0.97] < .001 ***Group 0 [−0.24, 0.24] .99Pretest 0.67 [0.53, 0.81] < .001 ***Group × pretest −0.27 [−0.47,−0.07] .0088 **The dependent variable is the posttest score*p < .05, ** p < .01, *** p < .001ReferencesAu, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi,S. M. (2015). Improving fluid intelligence with training on work-ing memory: A meta-analysis. Psychonomic Bulletin & Review,22(2), 366–377. https://doi.org/10.3758/s13423-014-0699-xBall, K., Edwards, J. D., & Ross, L. A. (2007). The impactof speed of processing training on cognitive and everydayfunctions. The Journals of Gerontology: Series B, 62, 19–31.https://doi.org/10.1093/geronb/62.special issue 1.19Baltes, P. B. (1987). Theoretical propositions of life-span devel-opmental psychology: On the dynamics between growthand decline. Developmental Psychology, 23(5), 611–626.https://doi.org/10.1037/0012-1649.23.5.611Baniqued, P., Kranz, M., Voss, M., Lee, H., Cosman, J., Severson,J., & Kramer, A. (2014). Cognitive training with casual videogames: Points to consider. Frontiers in Psychology, 4, 1010.https://doi.org/10.3389/fpsyg.2013.01010Bjorklund, D. F., & Douglas, R. N. (1997). The development ofmemory strategies. In N. Cowan, & C. Hulme (Eds.) , Thedevelopment of memory in childhood (pp. 201–246). Sussex, UK:Psychology Press.Bjorklund, D. F., & Schneider, W. (1996). The interactionof knowledge, aptitude, and strategies in children’s memoryperformance. In H. W. Reese (Ed.), Advances in child developmentand behavior (pp. 59–89). San Diego: Academic Press.Borella, E., Carbone, E., Pastore, M., Beni, R. D., & Carretti,B. (2017). Working memory training for healthy older adults:The role of individual characteristics in explaining short-andlong-term gains. Frontiers in Human Neuroscience, 11, 99.https://doi.org/10.3389/fnhum.2017.00099Brehmer, Y., Li, S. C., Mu¨ller, V., von Oertzen, T., & Lindenberger,U. (2007). Memory plasticity across the life span: Uncoveringchildren’s latent potential. Developmental Psychology, 43(2),456–478. https://doi.org/10.1037/0012-1649.43.2.465Burton, A., Altman, D. G., Royston, P., & Holder, R. L. (2006).The design of simulation studies in medical statistics. Statistics inMedicine, 25, 4279–4292. https://doi.org/10.1002/sim.2673Chan, J. S., Wu, Q., Liang, D., & Yan, J. H. (2015). Visu-ospatial working memory training facilitates visually-aidedexplicit sequence learning. Acta Psychologica, 161, 145–153.https://doi.org/10.1016/j.actpsy.2015.09.008Chesher, A. (1991). The effect of measurement error. Biometrika,78(3), 451–462. https://doi.org/10.1093/biomet/78.3.451Colom, R., Stein, J. L., Rajagopalan, P., Martinez, K., Hermel, D.,Wang, Y., & Thompson, P. M. (2013). Hippocampal structureand human cognition: Key role of spatial processing and evidencesupporting the efficiency hypothesis in females. Intelligence,41(2), 129–140. https://doi.org/10.1016/j.intell.2013.01.002Cox, B. D. (1994). Children’s use of mnemonic strategies: Variabilityin response to metamemory training. The Journal of GeneticPsychology, 155(4), 423–442. https://doi.org/10.1080/00221325.1994.9914792Dahlin, K. I. E. (2011). Effects of working memory training onreading in children with special needs. Reading and Writing,24(4), 479–491. https://doi.org/10.1007/s11145-010-9238-yEspejo, J., Day, E. A., & Scott, G. (2005). Performance evalua-tions, need for cognition, and the acquisition of a complex skill:An attribute-treatment interaction. Personality and Individual Dif-ferences, 38(8), 1867–1877. https://doi.org/10.1016/j.paid.2004.10.003Feng, J., & Spence, I. (2007). Effects of cognitive training onindividual differences. In D. Harris (Ed.), Engineering psychologyand cognitive ergonomics (pp. 279–287). Berlin: Springer.Foster, J. L., Harrison, T. L., Draheim, K. L. H. C., Redick, T. S.,& Engle, R. W. (2017). Do the effects of working memorytraining depend on baseline ability level? Journal of ExperimentalPsychology: Learning, Memory, and Cognition. https://doi.org/10.1037/xlm0000426.Galton, F. (1886). Regression towards mediocrity in hereditary stature.The Journal of the Anthropological Institute of Great Britain andIreland, 15, 246–263.Gatz, M., Svedberg, P., Pedersen, N. L., Mortimer, J. A., Berg, S.,& Johansson, B. (2001). Education and the risk of Alzheimer’sdisease: Findings from the study of dementia in Swedish twins.Journal of Gerontology: Psychological Sciences, 56B(5), 292–300. https://doi.org/10.1093/geronb/56.5.P292Gaultney, J. F., Bjorklund, D. F., & Goldstein, D. (1996).To be young, gifted, and strategic: Advantages for memoryperformance. Journal of Experimental Child Psychology, 61(1),43–66. https://doi.org/10.1006/jecp.1996.0002Gopher, D., Weil, M., & Siegel, D. (1989). Practice under changingpriorities: An approach to the training of complex skills. Acta Psy-chologica, 71(1–3), 147–177. https://doi.org/10.1016/0001-6918(89)90007-3Guye, S., Simoni, C. D., & von Bastian, C. C. (2017). Do individualdifferences predict change in cognitive training performance? Alatent growth curve modeling approach. Journal of CognitiveEnhancement, 1(4), 374–393. https://doi.org/10.1007/s41465-017-0049-9Hampstead, B. M., Stringer, A. Y., Stilla, R. F., Giddens, M., &Sathian, K. (2012). Mnemonic strategy training partially restoreshippocampal activity in patients with mild cognitive impairment.Hippocampus, 22(8), 1652–1658. https://doi.org/10.1002/hipo.22006Mem Cogn (2018) 46:1315–1330 1329Holmes, J., & Gathercole, S. E. (2013). Taking working memorytraining from the laboratory into schools. Educational Psychology.https://doi.org/10.1080/01443410.2013.797338Ihle, A., Oris, M., Fagot, D., Maggiori, C., & Kliegel, M. (2016).The association of educational attainment, cognitive level ofjob, and leisure activities during the course of adulthoodwith cognitive performance in old age: The role of opennessto experience. International Psychogeriatrics, 28(5), 733–740.https://doi.org/10.1017/S1041610215001933Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008).Improving fluid intelligence with training on working mem-ory. PNAS, 105(19), 6829–6833. https://doi.org/10.1073/pnas.0801268105Jausˇovec, N., & Pahor, A. (2017). Increasing intelligence. San Diego:Academic Press.Johns, G. (1981). Difference score measures of organizational behav-ior variables: A critique. Organizational Behavior and HumanPerformance, 27(3), 443–463. https://doi.org/10.1016/0030-5073(81)90033-7Karbach, J., & Kray, J. (2016). Executive functions. In T. Strobach, &J. Karbach (Eds.), Cognitive training (pp. 93–103). Switzerland:Springer.Karbach, J., & Unger, K. (2014). Executive control training frommiddle childhood to adolescence. Frontiers in Psychology, 5, 390.https://doi.org/10.3389/fpsyg.2014.00390Karbach, J., Strobach, T., & Schubert, T. (2015). Adaptiveworking-memory training benefits reading, but not mathematicsin middle childhood. Child Neuropsychology, 21(3), 285–301.https://doi.org/10.1080/09297049.2014.899336Kattenstroth, J. C., Kalisch, T., Holt, S., Tegenthoff, M., &Dinse, H. (2013). Six months of dance intervention enhancespostural, sensorimotor, and cognitive performance in elderlywithout affecting cardio-respiratory functions. Frontiers in AgingNeuroscience, 5, 5. https://doi.org/10.3389/fnagi.2013.00005Kliegel, M., Zimprich, D., & Rott, C. (2004). Life-long intellectualactivities mediate the predictive effect of early education oncognitive impairment in centenarians: A retrospective study. Aging& Mental Health, 8(5), 430–437. https://doi.org/10.1080/13607860410001725072Kliegl, R., Smith, J., & Baltes, P. B. (1990). On the locusand process of magnification of age differences duringmnemonic training. Developmental Psychology, 26(6), 894–904.https://doi.org/10.1037/0012-1649.26.6.894Kline, R. B. (2016). Principles and Practice of Structural EquationModeling (4th edn). New York: The Guilford Press.Klingberg, T. (2010). Training and plasticity of working memory.Trends in Cognitive Science, 14(7), 317–324. https://doi.org/10.1016/j.tics.2010.05.002Koller, D., & Friedman, N. (2009). Probabilistic graphical models.Principles and techniques. Cambridge: The MIT Press.Kramer, A. F., & Willis, S. L. (2002). Enhancing the cognitive vitalityof older adults. Current Directions in Psychological Science,11(5), 173–177. https://doi.org/10.1111/1467-8721.00194Lee, H., Boot, W. R., Baniqued, P. L., Voss, M. W., Prakash, R. S.,Basak, C., & Kramer, A. F. (2015). The relationship betweenintelligence and training gains is moderated by training strategy.PLoS One, 10(4). https://doi.org/10.1371/journal.pone.0123259Lee, H., Boot, W. R., Basak, C., Voss, M. W., Prakash, R. S., Neider,M., . . . , Kramer, A. F. (2012). Performance gains from directedtraining do not transfer to untrained tasks. Acta Psychologica, 139,146–158. https://doi.org/10.1016/j.actpsy.2011.11.003Loosli, S. V., Buschkuehl, M., Perrig, W. J., & Jaeggi, S. M.(2012). Working memory training improves reading processes intypically developing children. Child Neuropsychology, 18(1), 62–78. https://doi.org/10.1080/09297049.2011.575772Lord, F. M. (1956). The measurement of growth. Educa-tional and Psychological Measurement, 16(3), 421–437.https://doi.org/10.1002/j.2333-8504.1956.tb00058.xLo¨vde´n, M., Brehmer, Y., Li, S. C., & Lindenberger, U. (2012).Training-induced compensation versus magnification of individ-ual differences in memory performance. Frontiers in HumanNeuroscience, 6, 141. https://doi.org/10.3389/fnhum.2012.00141McArdle, J. J. (2009). Latent variable modeling of differencesand changes with longitudinal data. Annual Review of Psychol-ogy, 60(1), 577–605. https://doi.org/10.1146/annurev.psych.60.110707.163612McArdle, J. J., & Hamagami, F. (2001). Latent difference scorestructural models for linear dynamic analysis with incompletelongitudinal data. In L. Collins, & A. Sayer (Eds.), New methodsfor the analysis of change (pp. 137–175). Washington, D.C.:American Psychological Association.Morrison, A. B., & Chain, J. M. (2011). Does working memorytraining work? The promise and challenges of enhancing cognitionby training working memory. Psychonomic Bulletin & Review,18(1), 46–60. https://doi.org/10.3758/s13423-010-0034-0Pahor, A., & Jausˇovec, N. (2014). The effects of theta transcranialalternating current stimulation (tACS) on fluid intelligence.International Journal of Psychophysiology, 93(3), 322–331.https://doi.org/10.1016/j.ijpsycho.2014.06.015Polanı´a, R., Nitsche, M. A., Korman, C., Batsikadze, G., & Paulus,W. (2012). The importance of timing in segregated theta phase-coupling for cognitive performance. Current Biology, 22(14),1314–1318. https://doi.org/10.1016/j.cub.2012.05.021Raz, N. (2000). Aging of the brain and its impact on cognitiveperformance: Integration of structural and functional findings. InF. M. Craik, & T. A. Salthouse (Eds.), The handbook of aging andcognition (pp. 1–90). Mahwah: Lawrence Erlbaum Associates.Redick, T. S., Shipstead, Z., Harrison, T. L., Hicks, K. L., Fried,D. E., Hambrick, D. Z., . . . , Engle, R. W. (2013). No evidenceof intelligence improvement after working memory training: Arandomized, placebo-controlled study. Journal of ExperimentalPsychology, 142(2), 359–379. https://doi.org/10.1037/a0029082Roma´n, F. J., Lewis, L. B., Chen, C. H., Karama, S., Burgaleta,M., Martı´nez K, . . . , Colom, R. (2016). Gray matter respon-siveness to adaptive working memory training: A surface-basedmorphometry study. Brain Structure & Function, 221(9), 4369–4382. https://doi.org/10.1007/s00429-015-1168-7Santarnecchi, E., Brem, A. K., Levenbaum, E., Thompson, T., Cohen,K. R., & Pascual-Leone, A. (2015). Enhancing cognition usingtranscranial electrical stimulation. Current Opinion in BehavioralSciences, 4, 171–178. https://doi.org/10.1016/j.cobeha.2015.06.003Santarnecchi, E., Muller, T., Rossi, S., Sarkar, A., Polizzotto, N.,Rossi, A., & Kadosh, R. C. (2016). Individual differences andspecificity of prefrontal gamma frequency-tACS on fluid intel-ligence capabilities. Cortex, 75, 33–43. https://doi.org/10.1016/j.cortex.2015.11.003Schafer, J. L., & Graham, J. W. (2002). Missing data: Our viewof the state of the art. Psychological Methods, 7(2), 147–177.https://doi.org/10.1037//1082-989X.7.2.147Schmiedek, F., Lo¨vde´n, M., & Lindenberger, U. (2010). Hundreddays of cognitive training enhance broad cognitive abilities inadulthood: Findings from the COGITO study. Frontiers in AgingNeuroscience, 2, 27. https://doi.org/10.3389/fnagi.2010.00027Schneider, W. (2012). Memory development in childhood. In U.Goswami (Ed.), Handbook of childhood cognitive development(pp. 236–256). London: Blackwell.Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is workingmemory training effective? Psychological Bulletin, 138(4), 628–654. https://doi.org/10.1037/a00274731330 Mem Cogn (2018) 46:1315–1330Sorenson, H. (1933). Mental ability over a wide range of ages. Journalof Applied Psychology, 17, 729–741. https://doi.org/10.1037/h0072233Swanson, H. L. (2014). Does cognitive strategy training on wordproblems compensate for working memory capacity in childrenwith math difficulties? Journal of Educational Psychology,106(3), 831–848. https://doi.org/10.1037/a0035838Swanson, H. L. (2015). Cognitive strategy interventions improveword problem solving and working memory in childrenwith math disabilities. Frontiers in Psychology, 6, 1099.https://doi.org/10.3389/fpsyg.2015.01099Verhaeghen, P., & Marcoen, A. (1996). On the mechanisms of plas-ticity in young and older adults after instruction in the method ofloci: Evidence for an amplification model. Psychology and Aging,11(1), 164–178. https://doi.org/10.1037//0882-7974.11.1.164Wall, T. D., & Payne, R. (1973). Are deficiency scores deficient?Journal of Applied Psychology, 58(3), 322–326. https://doi.org/10.1037/h0036227Willis, S. L., & Nesselroade, C. S. (1990). Long-term effects of fluidability training in old-old age. Developmental Psychology, 26(6),905–910. https://doi.org/10.1037/0012-1649.26.6.905Zinke, K., Zeintl, M., Eschen, A., Herzog, C., & Kliegel, M.(2012). Potentials and limits of plasticity induced by workingmemory training in old-old age. Gerontology, 58(1), 79–87.https://doi.org/10.1159/000324240Zinke, K., Zeintl, M., Rose, N. S., Putzmann, J., Pydde, A., & Kliegel,M. (2014). Working memory training and transfer in olderadults: Effects of age, baseline performance, and training gains.Developmental Psychology, 50(1), 304–315. https://doi.org/10.1037/a0032982

Reviewed by:

Abstract

Extracted data

Reviewed by:

Abstract

Extracted data

Related items

Related items