From A to E: Analyzing TPC’s OLTP Benchmarks The obsolete, the ubiquitous, the unexplored

Introduced in 2007, TPC-E is the most recently standard-ized OLTP benchmark by TPC. Even though TPC-E has already been around for six years, it has not gained the pop-ularity of its predecessor TPC-C: all the published results for TPC-E use a single database vendor’s product. TPC-E is significantly different than its predecessors. Some of its distinguishing characteristics are the non-uniform input creation, longer-running and more complicated transactions, more difficult partitioning etc. These factors slow down the adoption of TPC-E. In turn, there is little knowledge in the community about how TPC-E behaves micro-architecturally and within the database engine. To shed light on TPC-E, we implement it on top of a scal-able open-source database engine, Shore-MT, and perform a workload characterization study, comparing it with the previous, much better known OLTP benchmarks of TPC: TPC-B and TPC-C. In parallel, we study the evolution of the OLTP benchmarks throughout the decades. Our results demonstrate that TPC-E exhibits similar micro-architectural behavior to TPC-B and TPC-C, even though it incurs less stall time and higher instructions per cycle. On the other hand, within the database engine it suffers more from logi-cal lock contention. Therefore, we argue that, on the hard-ware side, TPC-E needs less aggressive processors. Whereas on the software side it can benefit from designs based on intra-transaction parallelism, logical partitioning, and opti-mistic concurrency control to minimize the effects of lock contention without introducing distributed transactions. 1