Efficient Evolutionary Search of Attention Convolutional Networks via Sampled Training and Node Inheritance

The performance of deep neural networks is heavily dependent on its architecture and various neural architecture search strategies have been developed for automated network architecture design. Recently, evolutionary neural architecture search (EvoNAS) has received increasing attention due to the attractive global optimization capability of evolutionary algorithms. However, EvoNAS suffers from extremely high computational costs because a large number of performance evaluations are usually required in evolutionary optimization and training deep neural networks is itself computationally very expensive. To address this issue, this paper proposes a computationally efficient framework for evolutionary search of convolutional networks based on a directed acyclic graph, in which parents are randomly sampled and trained on each mini-batch of training data. In addition, a node inheritance strategy is adopted so that the fitness of all offspring individuals can be evaluated without training them. Finally, we encode a channel attention mechanism in the search space to enhance the feature processing capability of the evolved neural networks. We evaluate the proposed algorithm on the widely used datasets, in comparison with 30 state-of-the-art peer algorithms. Our experimental results show the proposed algorithm is not only computationally much more efficient, but also highly competitive in learning performance