An overview of the AutoPrognosis framework.

AutoPrognosis is a highly extensible AutoML framework built upon a plugin system. Based on the configured plugins for data imputation, preprocessing and classification, AutoPrognosis constructs an ML pipeline ensemble from the most performant pipelines developed with base classification plugins. (a) An example ensemble composed of three ML pipelines was illustrated to demonstrate the AutoML workflow of AutoPrognosis. All pipelines include four major procedures: imputation, preprocessing, classification, and calibration. In pipeline 1, the multivariate imputation by chained equations (MICE) plugin is applied for missing data imputation. The imputed data are then passed to fast ICA to create a compact, low-dimension data representation. The random forest classifier is used for the prediction task and its outputs are calibrated with a sigmoid function. Pipeline 2 and 3 are constructed in the same manner for the end-to-end prediction. AutoPrognosis first searches for the most performant ML pipelines among all possible combination of configured plugins. The selected pipelines are then combined as an ensemble model to achieve the best prediction performance. Two types of ensemble structure, i.e., stacked and weighted ensembles, are considered in AutoPrognosis, and Bayesian optimization is used to tune ensemble parameters for each structure. The optimal ensemble is selected based on the configured performance metric. Various explainer plugins of AutoPrognosis can be enabled for the ensemble to provide explanations along with the classification outputs. Detailed description of the algorithm can be found in [10–12]. (b) In this study, the UK CF dataset was provided as input to AutoPrognosis to search the optimal ML model for the composite endpoint prognostic task. The constructed ML model was a weighted ensemble of three ML pipelines. As illustrated in the calibration curves, the random forest pipelines tended to underestimate (above the dashed line) the risk level of high-risk patients. While the logistic regression pipeline was able to identify high-risk CF patients, its prognostic output was significantly higher than observed risks and would lead to many false alarms. AutoPrognosis was able to take advantage of all ML pipelines and create an optimal ensemble with the best prognostic accuracy.