Reliability estimation in regression supported with meta-learning and principal component analysis

Reliability estimation in regression supported with meta-learning and principal component analysis Reliability estimates for individual predictions can provide important risk-sensitive information. They enable users to distinguish between better and worse predictions which are very important when dealing with decision-critical prediction problems. Unfortunately, when used on different domains and models, reliability estimates perform differently. That is the reason why we require an approach that can foretell which estimate will work best with a specific domain/model pair. In our thesis we have experimented with meta-learning approach to automatically select the best estimate for a specific domain and regression model. We used seven different meta-classifiers on eight regression models and with nine reliability estimates. These reliability estimates represented the class values of the meta-classification process. During the creation of the metadata for meta-learning, we have chosen meta-features that we considered to be most appropriate for this problem. The results showed that the best performing meta-model was random forest. Meta-model neural networks gave the worst results. Additionally, we proposed principal component analysis approach for creation of two new reliability estimates as combinations of existing ones to see if these new estimates would perform. These results were also compared with the results of the existing nine estimates and the best performing meta-classifier. The results showed that the meta-classifier achieved the best results and the estimate BVCK the second best