A Framework for Pervasive Context Awareness

The proliferation of pervasive computing devices with unprecedented sensing, communication, and computation capabilities has enabled continuous and ubiquitous monitoring of users and their surrounding environments. While smartphones have evolved from only communication devices into powerful personal computing platforms, connected devices such as smartwatches, cameras, motions sensors, thermostats, and energy meters, collectively dubbed as the Internet of Things, are also rapidly permeating our living spaces. The continuous stream of richly annotated, real-time data made available by tapping into the spectrum of sensors available on these devices has led to the emergence of a sprawling ecosystem of context-aware apps. These apps use sensors such as GPS, microphone, accelerometer, and gyroscope to make diverse inferences about user activities and contexts. Typical context-aware apps employ a suite of machine learning algorithms to extract semantically meaningful inferences from sensor data. However, today's connected devices and mobile operating systems are not designed to support sensing and inference workloads. Compared with the rich network stack on pervasive devices, where one can leverage protocols and abstractions in different layers, context inference apps are composed in a monolithic fashion. That is, developers have to implement all data collection and inference logic using the raw data. As a result, the development and execution of context inference apps today are accompanied by myriad difficulties. First, the limited sensor coverage from running inferences on a single device and the need of hand-picking high-level features have made it challenging to achieve high inference accuracy in building the classification model of apps. Second, the continuous execution of complex algorithms on the main app processor of smartphones and the use of energy-hungry sensors have resulted in high energy consumption of inference apps, especially when they are executed on battery-powered pervasive devices. Finally, the monolithic development practice limits the adoption of inference apps onto heterogeneous devices, and requires enormous amount of efforts from app developers in composing the apps. In this dissertation we make three research contributions towards building a framework for pervasive context awareness:- We first showcase that context-aware inferences can benefit from heterogeneous connected devices such as smartwatches. We develop an example app to autonomously infer workout exercises of users from only sensors on a commercial smartwatch, achieving 90% classification accuracy for both cardio and weightlifting exercises while extending the watch battery life by up to 19 hours compared with prior approaches. - Having identified the problems from our example app, we then perform three optimizations of context inference apps. We have achieved (1) comparable inference accuracy as traditional models and acceptable latency using deep learning without hand-picking features; (2) up to 30x speed-up of deep learning tasks using mobile GPUs and up to 60% energy saving of off-loading inference tasks from the CPU to the DSP; and (3) up to 37% accuracy improvement and up to 67% less energy consumption for context-aware apps from watch-phone coordinations.- Finally, we close the loop by proposing the design and implementation of a programming framework for context inference apps, with a set of programming abstractions and an associated runtime. The framework helps reduce development tasks by up to 4.5x and source lines of code by up to 12x. It also tackles runtime challenges and achieves 3x better inference accuracy by handling environmental dynamics.With this work we have also created an open-source toolkit for developing and executing context inference apps using heterogeneous pervasive devices