Detecting and Combining Programming Patterns

This thesis explores detecting patterns in the most general interface to computers: source and assembly program code. Because writing computer programs correctly is so difficult, there is a large assortment of software engineering techniques devoted to making this process easier and more efficient. Therefore, despite the huge space of possible programs, most programs written by humans will exhibit some of the same patterns. Project Laika detects the data structures used by guest programs using unsupervised Bayesian learning. Using these data structures, it can detect viruses with considerably better accuracy than ClamAV, a leading industry solution. Using high-level features makes Laika considerably more resistant to polymorphic worms, because it requires them to preserve their high-level structure through their polymorphic transformations. Project Macho generates Java programs from a combination of natural language, example inputs, and a database of Java code. Unlike past natural language programming systems, which were basically verbose versions of Visual Basic, Macho allows users to specify (small) programs in high-level language and use examples to fill in the details. We were able to generate satisfactory solutions for basic versions of several core utilities, including ls and grep, even when the natural language inputs were varied substantially