Reducing computational complexity of convolutional neural networks through filter pruning

Poster presentation at "Exploring New Possibilities" 2022 Doctoral College Conference at University of Surrey, UK. Team Members: Dr Emily Corrigan-Kavanagh, Dr. Helen Cooper, Prof Mark D. Plumbley. Abstract: Convolutional neural networks (CNNs) have performed state-of-the-art in many real-life applications, including image and audio classification. However, the large size and the computational complexity of CNNs is a bottleneck to deploy them on hardware, particularly on resource-constrained devices such as smartphones. Moreover, when many intelligent devices are connected and communicating, the computation speed should be faster to act in real-time and make every connected device aware of the situation without any delay. Therefore, reducing the size and speed-up in computations of CNNs during inference is an essential task for efficiency. Apart from obtaining efficient CNNs during inference, reducing computations in CNNs may reduce CO2 emission during their training stage. Typically, training CNNs for more time generates more CO2, e.g. running GTX 1080 Ti hardware for two days generates CO2 equal to 5.18 kg, equivalent to driving an average car for 20Km. Therefore, it is crucial to reduce CNNs computational complexity for sustainable solutions. The literature shows that CNNs require a large number of parameters to solve complex non-linear tasks. However, once the CNNs are trained, there may be redundancy in their parameters, and it is possible to obtain a sub-network from the original network that gives similar performance as that of the original network at a reduced number of parameters. One of the methods to obtain sub-network is through filter pruning, where the filters or parameters learned in CNNs are explicitly removed from the original network based on their importance. This work aims to remove filters based on their similarity and assume that similar filters produce similar output, and hence, one of the filters can be removed. By removing similar filters, our proposed method achieves a 25% reduction in parameters and 27% speed-up in computations at marginal drop in accuracy compared to the original network.This work was supported by Engineering and Physical Sciences Research Council (EPSRC) Grant EP/T019751/1 "AI for Sound