Abstract Hypothesis testing is an important part of

Statistical Model Checking (SMC). It is typically used to verify statements of the form p> p0 or p < p0, where p is an unknown probability intrinsic to the system model and p0 is a given threshold value. Many techniques for this have been introduced in the SMC literature. We give a comprehensive overview and comparison of these tech-niques, starting by introducing a framework in which they can all be described. We distinguish between three classes of techniques, differing in what type of output correctness guarantees they give when the true p is very close to the threshold p0. For each technique, we show how to parametrise it in terms of quantities that are meaningful to the user. Having parametrised them con-sistently, we graphically compare the boundaries of their decision thresholds, and numerically compare the cor-rectness, power and efficiency of the tests. A companion website allows users to get more insight in the properties of the tests by interactively manipulating the parame-ters.