Succinct and Assured Machine Learning: Training and Execution

Contemporary datasets are rapidly growing in size and complexity. This wealth of data is providing a paradigm shift in various key sectors including defense, commercial, and personalized computing. Over the past decade, machine learning and related fields have made significant progress in designing rigorous algorithms with the goal of making sense of this large corpus of available data. Concerns over physical performance (runtime and energy consumption), reliability (safety), and ease-of-use, however, pose major roadblocks to the wider adoption of machine learning techniques. To address the aforementioned roadblocks, a popular recent line of research is focused on performance optimization and machine learning acceleration via hardware/software co-design and automation. This thesis advances the state-of-the-art in this growing field by advocating a holistic automated co-design approach which involves not only hardware and software but also the geometry of the data and learning model as well as the security requirements. My key contributions include: Co-optimizing graph traversal, data embedding, and resource allocation for succinct training and execution of Deep Learning (DL) models. The resource efficiency of my end-to-end automated solutions not only enables compact DL training/execution on edge devices but also facilitates further reduction of the training time and energy spent on cloud data servers. Characterizing and thwarting adversarial subspace for robust and assured execution of DL models. I build a holistic hardware/software/algorithm co-design that enables just-in-time defense against adversarial attacks. My proposed countermeasure is robust against the strongest adversarial attacks known to date without violating the real-time response requirement, which is crucial in sensitive applications such as autonomous vehicles/drones. Proposing the first efficient resource management framework that empowers coherent integration of robust digital watermarks/fingerprints into DL models. The embedded digital watermarks/fingerprints are robust to removal and transformation attacks and can be used for model protection against intellectual property infringement. Devising the first reconfigurable and provably-secure framework that simultaneously enables accurate and scalable DL execution on encrypted data. The proposed framework supports secure streaming-based DL computation on cloud servers equipped with FPGAs. Developing the first scalable framework that enables real-time approximation of multi-dimensional probability density functions for causal Bayesian analysis. The proposed solution adaptively fine-tunes the underlying latent variables to cope with the data dynamics as it evolves over time