Estimation of individual prediction reliability using sensitivity analysis of regression models

The dissertation discusses the reliability estimation of individual regression predictions. In contrast with average measures for the evaluation of model accuracy, the reliability estimates for individual predictions can provide additional information which could be beneficial for evaluating the usefullness of the prediction (medical diagnosis, financial and control applications). As a novelty, the dissertation proposes a method for reliability estimation of predictions, which is based on the sensitivity analysis approach and is independent of the regression model. New reliability estimates are compared with traditional or adapted reliability estimates. The problem of optimal reliability estimate selection based on the given problem domain and the regression model was also studied using metalearning and internal cross-validation approach. The testing was performed with 8 regression models, with larger number of benchmark problem domains and in a real domain from the area of medical prognostics. The results showed the potential of the proposed methodology in practice