Meta-programming for cross-domain tensor optimizations

International audienceMany modern application domains crucially rely on tensor operations. The optimization of programs that operate ontensors poses difficulties that are not adequately addressed by existing languages and tools. Frameworks such as TensorFlow offer good abstractions for tensor operations, but target a specific domain, i.e. machine learning, and theiroptimization strategies cannot easily be adjusted to other domains. General-purpose optimization tools such as Pluto andexisting meta-languages offer more flexibility in applying optimizations but lack abstractions for tensors. This workcloses the gap between domain-specific tensor languages and general-purpose optimization tools by proposing theTensor optimizations Meta-Language (TeML). TeML offers high-level abstractions for both tensor operations and looptransformations, and enables flexible composition of transformations into effective optimization paths. This compositionality is built into TeML’s design, as our formal language specification will reveal. We also show that TeML can expresstensor computations as comfortably as TensorFlow and that it can reproduce Pluto’s optimization paths. Thus,optimized programs generated by TeML execute at least as fast as the corresponding Pluto programs. In addition, TeMLenables optimization paths that often allow outperforming Pluto