A generative adversarial network approach for the attenuation correction in PET-MR hybrid imaging

Positron emission tomography (PET) provides functional images useful to track metabolic processes in the body and enables the diagnosis of several diseases. The technique is based on the use of radiotracers that emit positrons whose annihilation with electrons in the human body produces photons which travel away in almost anti-parallel directions. A ring of detectors is used to detect them and an event is counted when two detectors are activated within a time window (coincidence window). Each couple of detectors defines a line of response (LOR) to which events are associated. After the scan, a reconstruction algorithm transforms the acquired data into a map of activity in the patient’s body. Photons do not travel in vacuum but in human body, thus a correction for their attenuation is required. PET images are characterized by limited spatial resolution. In order to get morphological details to combine to functional ones, PET-CT (PET and computed tomography) and PET-MR (PET and magnetic resonance) systems have been developed. Linear attenuation coefficient maps are obtainable directly from the CT scan in the case of PET-CT by means of an accurate energy rescaling to 511 keV. Unfortunately, there is no straightforward technique to be used in PET-MR to derive the attenuation properties of tissues from MR signals. Plenty of techniques have been developed to address such kind of problem and in this work we explore an original approach based on deep neural networks. These could provide a boost in the direction of a data-driven algorithm for attenuation correction by using structural, T 1 weighted, MR images transformed into pseudo-CTs, i.e. images whose intensity values are similar to the ones expected in a CT image. Already implemented deep learning techniques to this purpose require paired data. Unfortunately, it is quite hard to obtain a big dataset of paired medical images, i.e. MR and CT images belonging to the same patient. To overcome this limitation, we chose to develop an approach based on a Generative Adversarial Network (GAN) trained on unpaired data. A GAN is a deep learning architecture composed by two neural networks, a generator and a discriminator, fighting against each other: the generator tries to map the input to the desired output and the discriminator tells if the generated output is good or not. In the training phase, the generator has to maximize the similarity to the desired output and the score provided by discriminator; the discriminator instead has to distinguish between the fakes, produced by the generator, and the original data. After the training phase, the generator is capable of mapping any point in the input space (MR images) to a point in the output space (pseudo-CT images). The generation of pseudo-CTs from MRs with an unpaired training set in the case here proposed has been approached by using a CycleGAN (with some ad-hoc developed modifications), characterized by the presence of four networks: two generators, the transformations from MR to CT domain (MR2CT) and vice versa (CT2MR) and two discriminators (fake CT vs. real CT, fake MR vs. real MR). A cyclic consistency constraint imposes that the whole cycle is the identity operator: MR ≈ MR2CT(CT2MR(MR)). This requirement, introduced in the loss function, guides the network training to generate not just an image but an image of the specific input patient. We collected a dataset of structural MR brain images coming from the Autism Brain Imaging Data Exchange (ABIDE: http://fcon_1000.projects.nitrc.org/indi/abide/) project and CT scans provided by the NeuroAnatomy and image Processing LABoratory (NAPLAB) of the IRCCS SDN (Naples, IT). We used these unpaired examples to train a CycleGAN-like network. Prior implementations of deep learning models for the generation of medical images require working on single slices of the acquired images, due to the availability of algorithms developed for 2D natural images and limitations in computing power. The proposed approach has been developed to work directly on 3D data. A registration step that aligns all images to approximately the same orientation has proved to be necessary due to the low number of training examples. In fact, no paired data is required, but in any case retrieving a brain CT from a MR is a major issue which needs a simplification at first stage. Structural similarity index computed between the generated output and the expected one shows satisfactory results. Despite a validation on a more populated dataset is needed to release the current requirements on the initial image alignment, the proposed approach opens to the perspective of using data driven methods to several processing pipelines on medical images, including data augmentation, segmentation and classification. Further investigation on the behaviour of the network in case of abnormalities in the images is required. An advantage of this technique with respect to other currently available procedures for attenuation correction in PET-MR is that it does not need any extra MR acquisition: only the standard diagnostic T 1 -weighted image is used and, due to the low computational cost, images are translated from the MR to the CT domain in a couple of seconds. Building a large collection of publicly available images could undoubtedly lead to avoiding some preprocessing steps and to achieve better overall results