Benchmarking EdgeAI platforms using Arduino Nano 33 BLE

Kurzfassung

An edge device is an endpoint of a network usually comprising some sensors attached to a microcontroller. These are the devices that collect or gather real-time data from their surroundings and send it further in the communication pipeline. Now to process this data locally rather than sending it to the cloud or a powerful machine for computing is where edge AI comes into play. In essence, running machine learning models locally on these resource-constrained devices by processing the incoming data is edge AI. In this work, we aim to explore the evaluation of various edge AI frameworks/platforms along the whole development process starting from the data preprocessing, all the way to ML model deployment. Now, there are multiple platforms that claim to achieve this functionality. Some by providing a web-based application while others require an installation on a local computer. Some platforms train the ML models over the cloud while others on the local PC. Theoretically, ML models with the same configuration trained over the same data could give similar results but how that compares on a spectrum of frameworks is what we want to explore. The primary goal is to determine which platform or framework delivers the best balance between computational efficiency and practical utility. To facilitate this, we benchmarked several edge AI frameworks across five criti-cal metrics like power consumption, memory usage, inference time, model accuracy, and user-friendly interface. The benchmarks include assessments of power (during both idle and active states), memory requirements, the time taken for each platform to perform a standard inference task, and accuracy measurements for a predefined human mo-tion-sense dataset. Additionally, usability was assessed based on the ease of integration,configuration, and modification provided by the platforms, considering a developer’s perspective.