Investigating the psychometric properties of the Suicide Stroop task

Behavioral measures are increasingly used to assess suicidal thoughts and behaviors. Some measures, such as the Suicide Stroop Task, have yielded mixed findings in the literature. An understudied feature of these behavioral measures has been their psychometric properties, which may affect the probability of detecting significant effects and reproducibility. In the largest investigation of its kind, we tested the internal consistency and concurrent validity of the Suicide Stroop Task in its current form, drawing from seven separate studies (N = 875 participants, 64% female, aged 12 to 81 years). Results indicated that the most common Suicide Stroop scoring approach, interference scores, yielded unacceptably low internal consistency (rs = -.09-.13) and failed to demonstrate concurrent validity. Internal consistency coefficients for mean reaction times (RTs) to each stimulus type ranged from rs = .93-.94. All scoring approaches for suicide-related interference demonstrated poor classification accuracy (AUCs = .52-.56) indicating that scores performed near chance in their ability to classify suicide attempters from nonattempters. In the case of mean RTs, we did not find evidence for concurrent validity despite our excellent reliability findings, highlighting that reliability does not guarantee a measure is clinically useful. These results are discussed in the context of the wider implications for testing and reporting psychometric properties of behavioral measures in mental health research