How to use replicate weights in health survey analysis using the National Nutrition and Physical Activity Survey as an example

Objective:To conduct nutrition-related analyses on large-scale health surveys, two aspects of the survey must be incorporated into the analysis: the sampling weights and the sample design; a practice which is not always observed. The present paper compares three analyses: (1) unweighted; (2) weighted but not accounting for the complex sample design; and (3) weighted and accounting for the complex design using replicate weights. Design:Descriptive statistics are computed and a logistic regression investigation of being overweight/obese is conducted using Stata. Setting:Cross-sectional health survey with complex sample design where replicate weights are supplied rather than the variables containing sample design information. Participants:Responding adults from the National Nutrition and Physical Activity Survey (NNPAS) part of the Australian Health Survey (2011-2013). Results:Unweighted analysis produces biased estimates and incorrect estimates of SE. Adjusting for the sampling weights gives unbiased estimates but incorrect SE estimates. Incorporating both the sampling weights and the sample design results in unbiased estimates and the correct SE estimates. This can affect interpretation; for example, the incorrect estimate of the OR for being a current smoker in the unweighted analysis was 1·20 (95 % CI 1·06, 1·37), t= 2·89, P = 0·004, suggesting a statistically significant relationship with being overweight/obese. When the sampling weights and complex sample design are correctly incorporated, the results are no longer statistically significant: OR = 1·06 (95 % CI 0·89, 1·27), t = 0·71, P = 0·480. Conclusions:Correct incorporation of the sampling weights and sample design is crucial for valid inference from survey data