The only thing constant in the fast-moving world of artificial intelligence (AI) is change. Almost as soon as a new model comes out, it seems like it is made obsolete by a competitor’s release. This blazing pace of progress gives us a steady stream of enhancements and new features to unlock better performance and greater levels of productivity all the time. But, despite its growing importance, TinyML applications are not progressing at the same rate.
Machine learning engineer Felix Galindo believes the reason we are not seeing as much innovation on tiny hardware platforms is not because the models perform poorly, but because existing infrastructure is lacking. Whereas cloud-based models receive frequent updates, TinyML models are typically frozen at deployment with no easy way to make updates. Galindo is trying to change this with an incremental model-update system called TinyMLDelta.
Roll with the changes
TinyMLDelta aims to solve one of embedded AI’s biggest pain points: the difficulty of updating machine learning models running on microcontrollers. Traditional over-the-air (OTA) updates require sending an entire TensorFlow Lite Micro model — often tens or even hundreds of kilobytes — to large numbers of devices. That consumes bandwidth, increases data costs, causes flash wear, and slows down iteration. As a result, most TinyML deployments remain stuck with their initial model, even as improvements become available.
The solution Galindo proposes is to send only the differences. Instead of replacing the whole model, TinyMLDelta generates a compact “patch” that modifies the model already stored on the device. In a real-world test using a 67-kilobyte sensor model, only 383 bytes changed between versions. The resulting patch weighed in at just 475 bytes, which is small enough to transmit cheaply and apply quickly even on the smallest MCUs.
But the system is more than a lightweight diff mechanism. Galindo emphasizes that guardrails are the most important part of the design. TinyMLDelta checks compatibility at multiple levels: interpreter ABI versions, operator sets, tensor I/O schemas, and required memory sizes. If the new model isn’t compatible with the existing firmware, TinyMLDelta automatically rejects the patch to avoid bricking devices. Updates use an A/B slot mechanism with crash-safe journaling to ensure that devices either fully succeed or roll back safely, even if power is lost mid-update.
The future of TinyML
TinyMLDelta currently supports TensorFlow Lite Micro and includes a full POSIX/macOS demo environment that simulates flash behavior. Planned additions include secure signing with SHA-256 and AES-CMAC, model versioning metadata, and OTA reference implementations for popular embedded platforms such as Zephyr, Arduino Uno R4 WiFi, and Particle’s Tachyon.
With billions of microcontrollers deployed worldwide and more edge-AI applications emerging every year, the ability to update models safely and efficiently may prove to be just as important as the models themselves. Galindo sees TinyMLDelta as an early but vital building block toward a full on-device AI lifecycle.