Scalable Inference in Graph-coupled Continuous-time Markov Chains

In this dissertation novel techniques for inference and learning of and decision-making in probabilistic graphical models over combinatorial state-spaces in continuous-time are developed. Such models are prevalent in the natural sciences and engineering. They can be used to describe various types of multi-agent dynamics on networks, with applications ranging from gene-regulatory networks in molecular biology to social networks in the social sciences to power-grids in engineering. All these examples exist in continuous-time. The first part of this thesis focuses on inference and learning from incomplete data, recorded at irregular positions in time - which represents the status-quo when dealing with data from molecular biology. Inference based on such data is intractable, with the most prominent reason being the exponentially growing state-space in the number of agents. A common approach to alleviate the state-space explosion are variational approximations. Variational approximations allow for exact computation of approximate inference solutions. This contrasts sampling, which computes approximate solutions of exact inference. Variational approximations exist in various forms for static- or a discrete-time graphical models. We provide a principled way of generating variational approximations for inference in continuous-time probabilistic graphical models of various degrees of accuracy. Similarly, learning probabilistic graphical models is non-tractable, as the number of graph-structures scales super-exponentially in the number of agents. This becomes especially hopeless in the aforementioned incomplete data-scenario, where for each of the possible graphs the intractable inference has to be performed. We relax the combinatorial problem of structure search to a gradient-based approach using optimization over mixtures of possible structures. By combination with the aforementioned variational inference, a scalable method for structure learning from incomplete time-series data is developed. The second part of this thesis focuses on decision-making in continuous-time multi-agent networks. First we take the perspective of an external policymaker. The goal is to perform optimal sequential interventional experiments to learn network structure and parameters requiring only a minimal amount of resources. For this, we adopt techniques from optimal experimental design and causal inference. Making optimal decisions under uncertainty requires accounting for all possible outcomes. This is intractable to do exactly in multi-agent systems. We develop a novel variational criterion that allows for making decisions in high-dimensional settings. We show how interventions from Pearls do-calculus can be translated from static to continuous-time models. The result is surprising - the outcome of an intervention in a continuous-time cyclic interaction graph is the same as in a static acyclic Bayesian network. Then, we take the perspective of individual agents and make optimal local state-dependent decisions in the form of actions to achieve a common goal. In this planning scenario, each agent has to account for all possible state-action trajectories of all other agents. Here, we leverage the variational perturbation theory developed in the first part of the thesis by making use of the duality between planning and inference. We show how the (discounted) planning via inference framework translates into continuous-time