Weakly supervised inference of personalized heart meshes based on echocardiography videos

Echocardiography provides recordings of the heart chamber size and function and is a central tool for non-invasive diagnosis of heart diseases. It produces high-dimensional video data with substantial stochasticity in the measurements, which frequently prove difficult to interpret. To address this challenge, we propose an automated framework to enable the inference of a high resolution personalized 4D (3D plus time) surface mesh of the cardiac structures from 2D echocardiography video data. Inferring such shape models arises as a key step towards accurate personalized simulation that enables an automated assessment of the cardiac chamber morphology and function. The proposed method is trained using only unpaired echocardiography and heart mesh videos to find a mapping between these distinct visual domains in a self-supervised manner. The resulting model produces personalized 4D heart meshes, which exhibit a high correspondence with the input echocardiography videos. Furthermore, the 4D heart meshes enable the automatic extraction of echocardiographic variables, such as ejection fraction, myocardial muscle mass, and volumetric changes of chamber volumes over time with high temporal resolution