A quantitative study of mutation and fault tolerance of pyro inference programs

Probabilistic programming allows users to model complex probability distributions and perform inference on such models. Since probabilistic reasoning and inference is a foundational technology of statistical learning programs, adoption of probabilistic programming systems has been growing in the past few years. Pyro is a commonly used probabilistic programming system written in Python that is based on the PyTorch deep learning framework and has become very popular for machine learning applications. However, since Pyro uses a deep learning framework to sample from distributions, there is a need to evaluate the approximate nature of computations and the resilience of probabilistic programs. Additionally, there is also a significant need to systemically test probabilistic programs to identify major errors. In this thesis, we systemically evaluate, test, and analyze Pyro probabilistic inference functions and programs. Since mutation testing is a well-established approach to test software against fault injections, we apply mutations to Pyro inference functions by using MutPy, a mutation testing tool for Python programs. Specifically, we use three popular inference programs as our testing suite and conduct mutation injection experiments on the Pyro inference library. Next, we analyze the data corruption and the amount of error introduced by mutations on inference programs. We provide a collective study of mutation tolerance of Pyro inference functions and programs where we also analyze mutation operators of MutPy and high-order mutations.U of I Onlyundergraduate senior thesis not recommended for open acces