Tiny-MLOps: a framework for orchestrating ML applications at the far edge of IoT systems

Empowering the Internet of Things devices with Artificial Intelligence capabilities can transform all vertical applications domains within the next few years. Current approaches favor hosting Machine Learning (ML) models on Linux-based single-board computers. Nevertheless, these devices’ cost and energy requirements limit the possible application scenarios. Conversely, today’s available 32-bit microcontrollers have much lower costs and only need a few milliwatts to operate, making them an energy-efficient and cost-effective alternative. However, the latter devices, usually referred to as far edge devices, have stringent resource constraints and host non-Linux-based embedded real-time operating systems. Therefore, orchestrating such devices executing portions of ML applications represents a major challenge with current tools and frameworks. This paper formally introduces the Tiny-MLOps framework as the specialization of standard ML orchestration practices, including far edge devices in the loop. To this aim, we will tailor each phase of the classical ML orchestration loop to the reduced resources available onboard typical IoT devices. We will rely on the proposed framework to deliver adaptation and evolving capabilities to resource-constrained IoT sensors mounted on an industrial rotary machine to detect anomalies. As a feasibility study, We will show how to programmatically re-deploy ML-based anomaly detection models to far edge devices. Our preliminary experiments measuring the system performance in terms of deployment, loading, and inference latency of the ML models will corroborate the usefulness of our proposal