Adapting branched networks to realise progressive intelligence

Progressive intelligence is a formulation of machine learning which trades-off performance requirements with resource availability. It does this by approaching the inference process incrementally. Current work in this area focuses on overall model performance rather than optimising its complete operating range. In this paper, we build upon existing explainability and branched neural network research to show how neural networks can be adapted to exhibit progressive intelligence. We assess the utility of joint branch optimisation for progressive intelligence using a number of explainability metrics. When optimising the area under curve of layerwise linear probe accuracy we find equally weighted early-exit branch optimisation produces models with the highest linear probe accuracy throughout the backbone. By varying confidence thresholds we represent the entire range over which the model can operate, we then explore its interaction with the scaling of the branched neural network backbone. Finally, we propose a novel ensemble inference strategy which utilises repeat predictions and requires no additional optimisation. Experiments with CIFAR10/100 show that this inference strategy can save up to 44% of the multiply accumulate operations used in inference whilst maintaining model performance, when compared against conventional early-exit methods