To send or not to send: weighing the costs and benefits of mailing an advance letter to participants before a telephone survey

Abstract Objective A letter was mailed to half the participants (Letter = 137; No Letter = 138) of a 5-year follow-up survey regarding smoking cessation before attempting contact for a telephone interview. The primary outcome was the number of completed surveys per group (response rate). Secondary analyses of the number of telephone calls placed and a cost analysis were performed. Results No conclusive effect was found on the response rates per group (59.1% Letter, 50.0% No Letter; p = 0.147). Additionally, a logistic regression, controlling for demographics, revealed that there was no direct effect of sending the letter on response rate (p = 0.369). Non-parametric analysis showed significantly fewer calls (U = 7962.5, z = − 2.274, p < 0.05 two-tailed) and significantly lower costs (U = 11112.00, z = 2.521, p < 0.05 two-tailed) in reaching participants in the Letter group. Mailing an advance letter to participants did not appear to effect response rates between the groups, even when controlling for demographics. However, further analysis examining the number of call attempts and the costs per group revealed the letter may have had other effects. These findings suggest that additional analyses may be merited when evaluating the effectiveness of methods to increase participation, such as an advance letter, especially in cases where the literature largely supports its effectual use. Trial registration ClinicalTrials.gov NCT03097445. Registered 31 March 201