Any computation is facilitated by some physical process, and the observable quantities of any physical process can be viewed as a computation. These close ties suggest that the study of what universal computers are capable of may lead to additional insight about the physical universe, and vice versa. In his thesis, we explore three lines of research that are linked to this central theme. The first partition shows how notions of non-computability and undecidability eventually led to evidence of emergence, the concept that even if a ‘theory of everything’ governing all microscopic interactions were discovered, the understanding of macroscopic order is likely to require additional insights. The second partition proposes a physically motivated ...