Efficient Neural Architecture Search with Multiobjective Evolutionary Optimization

Deep neural networks have become very successful at solving many complex tasks such as image classification, image segmentation, and speech recognition. These models are composed of multiple layers that have the capacity to learn increasingly higher-level features, without prior handcrafted specifications. However, the success of a deep neural network relies on finding the proper configuration for the task in hand. Given the vast number of hyperparameters and the massive search space, manually designing or fine-tuning deep learning architectures requires extensive knowledge, time, and computational resources. There is a growing interest in developing methods that automatically design a neural network´s architecture, known as neural architecture search (NAS). NAS is usually modeled as a single-objective optimization problem where the aim is to find an architecture that maximizes the prediction´s accuracy. However, most deep learning applications require accurate as well as efficient architectures to reduce memory consumption and enable their use in computationally-limited environments. This has led to the need to model NAS as a multiple objective problem that optimizes both the predictive performance and efficiency of the network. Furthermore, most NAS framework have focused on either optimizing the micro-structure (structure of the basic cell), or macro-structure (optimal number of cells and their connection) of the architecture. Consequently, manual engineering is required to find the topology of the non-optimized structure. Although NAS has demonstrated great potential in automatically designing an architecture, it remains a computationally expensive and time-consuming process because it requires training and evaluating many potential configurations. Recent work has focused on improving the search time of NAS algorithms, but most techniques have been developed and applied only for single-objective optimization problems. Given that optimizing multiple objectives has a higher complexity and requires more iterations to approximate the Pareto Front, it is critical to investigate algorithms that decrease the search time of multiobjective NAS. One critical application of deep learning is medical image segmentation. Segmentation of medical images provides valuable information for various critical tasks such as analyzing anatomical structures, monitoring disease progression, and predicting patient outcomes. Nonetheless, achieving accurate segmentation is challenging due to the inherent variability in appearance, shape, and location of the region of interest (ROI) between patients and the differences in imagining equipment and acquisition protocols. Therefore, neural networks are usually tailored to a specific application, anatomical region, and image modality. Moreover, medical image data is often volumetric requiring expensive 3D operations that result in large and complex architectures. Hence, training and deploying them requires considerable storage and memory bandwidth that makes them less suitable for clinical applications. To overcome these challenges, the main goal of this research is to automatically design accurate and efficient deep neural networks using multiobjective optimization algorithms for medical image segmentation. The proposed research consists of three major objectives: (1) to design a deep neural network that uses a multiobjective evolutionary based algorithm to automatically adapt to different medical image datasets while minimizing the model’s size; (2) to design a self-adaptive 2D-3D Fully Convolutional network (FCN) ensemble that incorporates volumetric information and optimizes both the performance and the size of the architecture; and (3) to design an efficient multiobjective neural architecture search framework that decreases the search time while simultaneously optimizing the micro- and macro-structure of the neural architecture. For the first objective, a multiobjective adaptive convolutional neural network named AdaResU-Net is presented for 2D medical image segmentation. The proposed AdaResU-Net is comprised of a fixed architecture and a learning framework that adjusts the hyperparameters to a particular training dataset using a multiobjective evolutionary based algorithm (MEA algorithm). The MEA algorithm evolves the AdaResU-Net network to optimize both the segmentation accuracy and model size. In the second objective, a self-adaptive ensemble of 2D-3D FCN named AdaEn-Net is proposed for 3D medical image segmentation. The AdaEn-Net is comprised of a 2D FCN that extracts intra-slice and long-range 2D context, and a 3D FCN architecture that exploits inter-slice and volumetric information. The 2D and 3D FCN architectures are automatically fitted for a specific medical image segmentation task by simultaneously optimizing the expected segmentation error and size of the network using the MEA algorithm. Finally, for the third objective, an efficient multiobjective neural architecture search framework named EMONAS is presented for 3D medical image segmentation. EMONAS has two main components, a novel search space that includes the hyperparameters that define the micro- and macro-structure of the architecture, and a Surrogate-assisted multiobjective evolutionary based algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values using a Random Forest surrogate and guiding selection probabilities. The broader impact of the proposed research is as follows: (1) automating the design of deep neural networks’ architecture and hyperparameters to improve the performance and efficiency of the models; and (2) increase the accessibility of deep learning to a broader range of organizations and people by reducing the need of expert knowledge and GPU time when automatically designing deep neural networks. In the medical area, the proposed models aim to improve the automatic extraction of data from medical images to potentially enhance diagnosis, treatment planning and survival prediction of various diseases such as cardiac disease and prostate cancer. Although the proposed techniques are applied to medical image segmentation tasks, they can also be implemented in other applications where accurate and resource-efficient deep neural networks are needed such as autonomous navigation, augmented reality and internet-of-things