Precision-recall curve for random forest model.

BackgroundTreatment of surgical pain is a common reason for opioid prescriptions. Being able to predict which patients are at risk for opioid abuse, dependence, and overdose (opioid-related adverse outcomes [OR-AE]) could help physicians make safer prescription decisions. We aimed to develop a machine-learning algorithm to predict the risk of OR-AE following surgery using Medicaid data with external validation across states.MethodsFive machine learning models were developed and validated across seven US states (90–10 data split). The model output was the risk of OR-AE 6-months following surgery. The models were evaluated using standard metrics and area under the receiver operating characteristic curve (AUC) was used for model comparison. We assessed calibration for the top performing model and generated bootstrap estimations for standard deviations. Decision curves were generated for the top-performing model and logistic regression.ResultsWe evaluated 96,974 surgical patients aged 15 and 64. During the 6-month period following surgery, 10,464 (10.8%) patients had an OR-AE. Outcome rates were significantly higher for patients with depression (17.5%), diabetes (13.1%) or obesity (11.1%). The random forest model achieved the best predictive performance (AUC: 0.877; F1-score: 0.57; recall: 0.69; precision:0.48). An opioid disorder diagnosis prior to surgery was the most important feature for the model, which was well calibrated and had good discrimination.ConclusionsA machine learning models to predict risk of OR-AE following surgery performed well in external validation. This work could be used to assist pain management following surgery for Medicaid beneficiaries and supports a precision medicine approach to opioid prescribing.