7 A Scalable Software Framework for Stateful Stream Data Processing on Multiple GPUs and Applications

Abstract During the past few years the increase of computational power has been realized using more processors with multiple cores and specific processing units like Graphics Processing Units (GPUs). Also, the introduction of program-ming languages such as CUDA and OpenCL makes it easy, even for non-graphics programmers, to exploit the computational power of massively parallel processors available in current GPUs. Although CUDA and OpenCL relieve programmers from considering many low-level details of parallel programming on multiple cores on a single GPU, the same support at a higher level of parallelization for multiple GPUs is still under research. In particular, fundamental issues of memory management and synchronization must be dealt with directly by the programmer. In this chapter we introduce concepts for CUDA-based frameworks which are de-signed for stateful stream data processing for graph-like arrangements of pro-cessing modules on two or more GPUs in a single compute node. We evaluate these concepts and further elaborate on the approach of our choice. Our approach relieves the programmer from error-prone chores of memory management and synchronization. The chapter presents detailed evaluation results which demon-strate the scalability of the proposed framework. To demonstrate the usability of our framework, we utilize it for demanding on-line processing in the areas of crys-tallographic structure detection and video decryption