StreamJIT: A Commensal Compiler for High-Performance Stream Programming

There are domain-specific libraries for many domains, enabling rapid and cost-effective de-velopment of complex applications. On the other hand, domain-specific languages are rare despite the performance advantages of compilation. We believe the reason is the multiple orders-of-magnitude higher cost of building a compiler compared to building a library. We propose commensal compilation, a new strategy for compiling embedded domain-specific languages by reusing the massive investment in modern language virtual machine plat-forms. Commensal compilers use the host language's front-end, use an autotuner instead of optimization heuristics, and use host platform APIs that enable back-end optimizations by the host platform JIT. The cost of implementing a commensal compiler is only the cost of implementing the domain-specific optimizations. We demonstrate the concept by im-plementing a commensal compiler for the stream programming language StreamJIT atop the Java platform. The StreamJIT commensal compiler takes advantage of the structure of stream programs to find the right amount of parallelism for a given machine and program. Our compiler achieves 2.4 times better performance than StreamIt's native code (via GCC