Improving Neural Architecture Search by Mixing a FireFly algorithm with a Training Free Evaluation

International audienceNeural Architecture Search (NAS) algorithms areused to automate the design of deep neural networks. Finding thebest architecture for a given dataset can be time consuming sincethese algorithms have to explore a large number of networks, andscore them according to their performances to choose the mostappropriate one. In this work, we propose a novel metric thatuses the Intra-Cluster Distance (ICD) score to evaluate the abilityof an untrained model to distinguish between data in order toapproximate its quality. We also use an improved version of theFireFly algorithm, more robust to the local optimums problemthan the baseline FireFly algorithm, as a search technique tofind the best neural network model adapted to a specific dataset.Experimental results on the different NAS Benchmarks show thatour metric is valid for either scoring CNNs and RNNs, and thatour proposed FireFly algorithm can improve the result obtainedby the state-of-art training-free method