Advice on Reliability Analysis with Small Samples - Revised Version

Whilst it is common statistical advice not to attempt a reliability analysis with a sample size less than 300 a recent simulation study by Yurdugül indicates that this is possible in certain circumstances. The most common statistic used in reliability analysis is Cronbach’s alpha and an often quoted rule of thumb is a coefficient value above 0.7 is acceptable for psychological constructs. However, Cortina found that the size of a Cronbach’s alpha coefficient depends upon the number of items in the scale with scales with more items having higher coefficients. The advantage of carrying out a reliability analysis is that it can enable a researcher to treat a group of variables on the same subject as a single scale variable, reducing the complexity of further analysis and reducing the risk of Type I errors. However, student researchers often find it hard to obtain sample sizes of 300. The purpose of this worksheet is to advise students about how to go about trying to validate a scale with smaller sample sizes