Deep intelligence as a service: A real-time scheduling perspective

This thesis presents a new type of cloud service, called {\em deep intelligence as a service\/}, that is expected to become increasingly common in the near future to support emerging "smart" embedded applications. This work offers a real-time scheduling model motivated by the special needs of embedded applications that use this service. A simple run-time scheduler is proposed for the server and prove an approximation bound in terms of application-perceived service utility. The service is implemented on representative device hardware and tested with a machine vision application illustrating the advantages of our scheme. The work is motivated by the proliferation of increasingly ubiquitous but resource-constrained embedded devices (often referred to as the Internet of Things -- IoT -- devices) and the growing desire to endow them with advanced interaction capabilities, such as voice recognition or machine vision. The trend suggests that machine intelligence will be increasingly offloaded to cloud or edge services that will offer advanced capabilities to the otherwise simple devices. New services will feature farms of complex trainable (or pre-trained) classifiers that client-side applications can send data to in order to gain certain types of advanced functionality, such as face recognition, voice command recognition, gesture recognition, or other. These new services will revolutionize human interaction with their physical environment, but may impose interesting real-time scheduling challenges in order to maintain responsiveness while maximizing service quality. This work includes challenges, designs for an efficient real-time scheduling algorithm for the new machine intelligence service, and evaluation on an implemented prototype