Mobile Deep Neural Network Inference in Edge Computing with Resource Restrictions

Recent advances in deep neural networks (DNNs) have substantially improved the accuracy of intelligent applications. However, the pursuit of a higher accuracy has led to an increase in the complexity of DNNs, which inevitably increases the inference latency. For many time-sensitive mobile inferences, such a delay is intolerable and could be fatal in many real-world applications. To solve this problem, one effective scheme known as DNN partition is proposed, which significantly improves the inference latency by partitioning the DNN to a mobile device and an edge server to jointly process the inference. This approach utilises the stronger computing capacity of the edge while reducing the data transmission. Nevertheless, this approach requires a reliable network connection, which is oftentimes unstable. Therefore, DNN partition is vulnerable in the presence of service outages. In this thesis, we are motivated to investigate how to maintain the quality of the service during service outages to avoid interruptions. Inspired by the recently developed early exit technique, we propose three solutions: (1) When the service outage time is predictable, we propose eDeepSave to decide which frames to process during the service outage. (2) When the service outage time is not predictable but relatively short, we design LEE to effectively learn the optimal exit point in a per-instance manner. (3) When the service outage time is not predictable and relatively long, we present the DEE scheme to learn the optimal action (to exit or not) at each exit point, so that the system can dynamically exit the inference by utilising the observed environmental information. For each scheme, we provide detailed mathematical proofs of the performance and then test their performance in real-world experiments as well as the extensive simulations. The results of the three schemes demonstrate their effectiveness in maintaining the service during the service outage under a variety of scenarios