Enhancing Risk Aware Decision in Healthcare through Probabilistic Modeling of Uncertainty

In domains with high stakes, like healthcare and medicine, trustworthy and robust decision-making is crucial due to the potential risks associated with misclassification. However, many traditional machine learning classifiers lack calibrated predictions, and reliable uncertainty estimates for new unseen data. This paper addresses the challenge of uncertainty quantification in text classification in healthcare and proposes a three-fold approach to \textit{support} robust and trustworthy decision-making by medical practitioners. To evaluate our solution, we implement it on a multi-label medical transcription dataset from Kaggle. Our study demonstrates three significant results: the ability of our model to reject uncertain predictions by providing a null set, the provision of set predictions with guaranteed coverage for further investigation, and the prioritization of decision-making based on confidence levels of predictions with the same label. Additionally, we tackle the issue of imbalanced datasets in the medical domain by employing the Mondrian Conformal Predictor with a Naïve Bayes classifier. Our findings are expected to enhance the risk-aware decision-making process in the medical field.KDD'23 Southern California Data Science Day, Applied Data Science (ADS) Track, Long Beach, C