Comparison of two models predicting IVF success; the effect of time trends on model performance

How well does the recently developed UK model predicting the success rate of IVF treatment (the 2011 Nelson model) perform in comparison with a UK model developed in the early 1990s (the Templeton model)? Both models showed similar performance, after correction for the increasing success rate over time of IVF. For counselling couples undergoing IVF treatment it is of paramount importance to be able to predict success. Several prediction models for the chance of success after IVF treatment have been developed. So far, the Templeton model has been recommended as the best approach after having been validated in several independent patient data sets. The Nelson model, developed in 2011 and characterized by the largest development sample containing the most recently treated couples, may well perform better. We tested both models in couples that were included in a national cohort study carried out in the Netherlands between the beginning of January 2002 and the end of December 2004. We analysed the IVF cycles of Dutch couples with primary infertility (n 5176). The chance of success was calculated using the two UK models that had been developed using the information collected in the Human Fertilisation and Embryology Authority database. Women were treated in 19911994 (Templeton) or 20032007 (Nelson). The outcome of success for both UK models is the occurrence of a live birth after IVF but the outcome in the Dutch data is an ongoing pregnancy. In order to make the outcomes compatible, we used a factor to convert the chance of live birth to ongoing pregnancy and use the overall terms success or no success after IVF. The discriminative ability and the calibration of both models were assessed, the latter before and after adjustment for time trends in IVF success rates. The two models showed a similarly limited degree of discriminative ability on the tested data (area under the receiver operating characteristic curve 0.597 for the Templeton model and 0.590 for the Nelson model). The Templeton model underestimated the success rate (observed 21 versus predicted 14); the Nelson model overestimated the success rate (observed 21 versus predicted 29). When the models were adjusted for the changing success rates over time, the calibration of both models considerably improved (Templeton observed 21 versus predicted 20; Nelson observed 21 versus predicted 24). We could only test the models in couples with primary infertility because detailed information on secondary infertile couples was lacking in the Dutch data. This shortcoming may have negatively influenced the performance of the Nelson model. The changes in success rates over time should be taken into account when assessing prediction models for estimating the success rate of IVF treatment. In patients with primary infertility, the choice to use the Templeton or Nelson model is arbitrary