A Lightweight Tool for Random Testing of Stream Processing

With the rise of Big Data technologies, distributed stream processing systems have gained popu-larity in the last years. These are used to continuously process streams of data from various sources, like customer activity or sensors measurements, to obtain an always up to date view of the data that allows the system to react on time to events. Stream processing systems, like any other system dealing with time and events, are hard to test. But we can find several proposals in the literature for dealing with those problems. In particular we focus on Pnueli’s approach based on the use of temporal logic for testing reactive systems. Our final goal is facilitating the adoption of temporal logic as an every day tool for the development of stream processing programs. To do that, we consider property-based testing (PBT), a random testing technique that has gained popularity in the software development industry, as a bridge between formal logic and software development practices like test driven de-velopment. As PBT only handles finite test cases, we propose a novel discrete time linear temporal logic for finite words. In order to increase its expressiveness for formulating stricter properties, in this logic we associate a timeout to each temporal connective, that determines the maximum time at which the corresponding sub-formulas should be solved to true for the whole formula to hold. We implement our logic as a library extending ScalaCheck for testing Spark Streaming programs