Real-Time Performance Benchmarking of TinyML Models in Embedded Systems (PICO: Performance of Inference, CPU, and Operations)
This provides actionable guidance for optimizing TinyML deployments in embedded systems, though it is incremental as it applies existing benchmarking methods to new platforms and models.
The paper tackles the problem of benchmarking TinyML models on embedded systems by developing PICO-TINYML-BENCHMARK, a framework that evaluates metrics like inference latency and CPU utilization. Results show the BeagleBone AI64 has consistent inference latency for AI tasks, while the Raspberry Pi 4 excels in resource efficiency and cost-effectiveness.
This paper presents PICO-TINYML-BENCHMARK, a modular and platform-agnostic framework for benchmarking the real-time performance of TinyML models on resource-constrained embedded systems. Evaluating key metrics such as inference latency, CPU utilization, memory efficiency, and prediction stability, the framework provides insights into computational trade-offs and platform-specific optimizations. We benchmark three representative TinyML models -- Gesture Classification, Keyword Spotting, and MobileNet V2 -- on two widely adopted platforms, BeagleBone AI64 and Raspberry Pi 4, using real-world datasets. Results reveal critical trade-offs: the BeagleBone AI64 demonstrates consistent inference latency for AI-specific tasks, while the Raspberry Pi 4 excels in resource efficiency and cost-effectiveness. These findings offer actionable guidance for optimizing TinyML deployments, bridging the gap between theoretical advancements and practical applications in embedded systems.