Versatile Benchmarking for Concurrent Production System Architectures

The shortage of adequate benchmarking facilities is a major problem in the evaluation of novel production system machine organizations. This paper presents a survey of benchmark programs used in published research for improvement of production systems. We offer a new benchmark problem that allows independent variation in the size of the database, the number of productions, the ratio between local and global data, and the variance in the size of local data clusters. This new benchmark, available via the Internet through anonymous ftp, is based on the traditional Traveling Salesperson Problem and has the advantage of being both simple and versatile. The advantages of our benchmark are explored through the evaluation of performance of a new production system architecture. Our experiments indicated that the performance of this new architecture scales with the size of the database, but is degraded by a database with high variance in the size of the local data clusters. 1 Introduction In spi..