Deep learning classification of cardiomegaly using combined imaging and non-imaging ICU data

In this paper, we investigate the classification of cardiomegaly using multimodal data, combining imaging data from chest radiography with routinely collected Intensive Care Unit (ICU) data comprising vital sign values, laboratory measurements, and admission metadata. In practice a clinician would assess for the presence of cardiomegaly using a synthesis of multiple sources of data, however, prior machine learning approaches to this task have focused on chest radiographs only. We show that non-imaging ICU data can be used for cardiomegaly classification and propose a novel multimodal network trained simultaneously on both chest radiographs and ICU data. We compare the predictive power of both single-mode approaches with the joint network. We use a subset of data from the publicly available MIMIC-CXR and MIMIC-IV datasets, which contain both chest radiographs and non-imaging ICU data for the same patients. The approach from non-imaging ICU data alone achieves an AUC of 0.684 and the standard chest radiography approach an AUC of 0.840. Our joint model achieves an AUC of 0.880. We conclude that non-imaging ICU data have predictive value for cardiomegaly, and that combining chest radiographs with non-imaging ICU data has the potential to improve model performance for the same subset of patients, with further work required to demonstrate a significant improvement