CodeReef and MLPerf demo: keeping up with AI, ML and systems innovation with portable workflows and collaborative benchmarking

We demonstrate how to use our open CodeReef.ai platform with open-source CodeReef solutions - a new way to share ML models as non-virtualized, portable, customizable and reusable packages. Such packages contain portable workflows with JSON input/output, CLI and Python API, software detection plugins, meta packages and pre-/post-processing scripts to automatically build, benchmark, test and customize ML-based applications across diverse platforms, environments, AI/ML frameworks, libraries, compilers and data sets. We will help participants to test several real-world CodeReef solutions to automatically build, reproduce, collaboratively benchmark and compare several recent MLPerf inference submissions across a wide range of platforms from Raspberry Pi and Android phones to HPC servers with powerful GPUs. Our long-term goal is to help the community keep track of AI, ML and systems innovation and facilitate research by sharing ML models and all related research artifacts along with research papers as production-ready CodeReef solutions which can be immediately used for collaborative benchmarking and reproducible comparison of ML, software and hardware stacks