Maintaining safe memory for security, debugging, and multi -threading

As transistor budgets grow enabling chip multi-core processors, adding hardware support to ensure the correctness and security of programs will be just as important, for the average user, as adding another core. The goal of our research is to develop hardware support for software checks and for multi-threaded synchronization that protects memory from corruption. This is the source of a significant number of bugs and security issues. We want efficient low-overhead run-time performance of these checks and synchronization so that it can be left on all of the time, even in production code releases. Bounds checking protects pointer and array accesses. It is the process of keeping track of the address boundaries for objects, and checking that the loads and stores do not stray outside bounds. It can serve two purposes: it can assist debugging by precisely capturing invalid memory accesses, and it can guarantee protection against buffer overflow attacks. Unfortunately, the high performance overhead of runtime bounds checking has prevented its inclusion in most released software. In this thesis, we analyze the sources of bounds checking overhead. We then consider hardware/software enhancements: the effect merging the check into a single instruction, software optimizations based on potential for over to eliminate checks, and changes to meta-data layout to limit copying overhead. Transactional Memories enable programmers to greatly simplify multithreaded code by eliminating lock synchronization and its attending deadlock and livelock problems. Unlike locks they also enable speculative concurrent execution through the critical section. Specialized transactional memory can also aid concurrent programming models by providing determinism only when needed at run-time. A key limitation of past transactional memory proposals are that they have a finite memory capacity. Virtualized transactional memory (unbounded in space and time) are desirable, as they can increase the scope of transactions' use, and thereby further simplify a programmer's job. In this thesis, we introduce Page-based Transactional Memory to support unbounded transactions. We combine transaction bookkeeping with the virtual memory system to support fast transaction conflict detection, commit, abort, and to maintain transactions' speculative dat