IN Brief:
- Ambiq has released heliaAOT 1.0, an ahead-of-time compiler for AI inference on Apollo processors.
- Internal Apollo510 benchmarks showed up to 76% lower latency and 72% lower energy per inference than LiteRT Micro under matched conditions.
- The compiler supports 65 operators and targets 15 Apollo SoCs, with FP16, FP32, and streaming neural network capabilities.
Ambiq has released heliaAOT 1.0, an ahead-of-time compiler designed to reduce the processing time, energy consumption, and firmware storage required for AI inference on its Apollo semiconductor platforms. Internal benchmarks performed on an Apollo510 device showed up to 76% lower inference latency and 72% lower energy per inference compared with LiteRT Micro under matched test conditions.
The compiler forms part of Ambiq's HELIA AI development environment and converts trained neural network models into optimised C code for execution on supported processors. It is intended for devices with limited memory and energy resources, including wearable equipment, biosignal monitors, industrial sensors, and other embedded systems that perform calculations locally.
AI inference on a microcontroller differs from the processing performed on larger computing platforms. A compact embedded device may have relatively little memory available for model weights, intermediate results, application software, and communications functions. Its processor must also share the available energy budget with sensing hardware and other electronic components.
These limits influence which neural networks can be deployed and how often they can execute. A model that produces an accurate result on a desktop computer may require substantial changes before it can operate within the processing and memory constraints of an embedded device.
Software execution contributes to the overall cost of inference. An interpreter-based runtime processes a model through a collection of supported operations, providing flexibility across different network structures. An ahead-of-time compiler can instead generate code tailored to the selected model and target hardware, reducing some of the work associated with interpreting operations during execution.
HeliaAOT performs this compilation before the model is deployed. The resulting C code can be incorporated into the embedded application and compiled for a supported Apollo device. The software can optimise the implementation around the operations used by the model, although performance remains dependent on the network architecture and the processor selected.
Ambiq measured the new compiler against LiteRT Micro on Apollo510 silicon operating at 96MHz. The tests used matched conditions for each model and examined latency, gross energy consumption on the microcontroller power rail, and flashed firmware image size.
The largest reported improvements occurred in the health monitoring workload group, which included electrocardiogram, photoplethysmography, and sleep-related models. Ambiq recorded 76% lower latency, 72% lower energy per inference, and a 56% smaller firmware image than the corresponding LiteRT Micro implementations.
Vision workloads produced reductions of 61% in latency and 51% in energy consumption, while voice and audio workloads showed improvements of 52% and 45% respectively. Tests using MLPerf Tiny reference models reported 36% lower latency and 27% lower energy per inference. Ambiq specifies that the latter results were based on the reference models and did not constitute an official MLPerf benchmark submission.
These comparisons describe the execution of selected models rather than the total energy consumption of completed products. A wearable monitor, for example, may spend significant energy acquiring signals, operating its wireless interface, and maintaining other functions. A 72% reduction in energy for one inference workload therefore does not establish an equivalent improvement in device battery life.
Firmware storage also requires separate interpretation. Ambiq reports median reductions in flashed firmware image size ranging from 29% to 56% across the measured groups. The figures describe storage occupied by the compiled software, not peak runtime memory consumption. The company's supporting notes exclude stack and heap requirements from the reported model-buffer measurements.
Compiler support has expanded from 35 to 65 operators over the preceding 12 months. Additional operators allow a wider selection of neural network architectures to be compiled without rewriting parts of the model. Developers can also implement custom operators when the required functionality is not included in the standard set.
Floating-point support has been extended to FP16 and FP32 models, allowing supported networks to run without conversion to integer-quantised representations. Quantisation can reduce memory and computation requirements, but changes numerical precision and may require further evaluation of model accuracy.
Retaining floating-point calculations can simplify deployment for selected workloads, although the associated processing cost still depends on the model and target hardware. Ambiq reports approximately fourfold faster FP32 inference than LiteRT Micro FP32 in a separate matched comparison covering six model families.
Streaming neural networks receive additional support through long short-term memory, gated recurrent unit, and singular value decomposition filter operations. These methods are suited to processing sequential information, including audio and physiological measurements, where the relationship between successive samples influences the result.
HeliaAOT can now target 15 Apollo systems-on-chip, compared with eight previously. Ambiq states that supported models can be retargeted through a single compiler configuration change, although differences in available memory and processing resources still need to be considered when moving an application between devices.
The company's heliaPROFILER tool provides a means of measuring model execution on development hardware. Such measurements are necessary when the application's behaviour differs from the benchmark workloads, particularly where the inference model operates alongside sensor acquisition, communications, or other real-time functions.
Ambiq says heliaAOT technology has already been deployed in mass-produced products and that additional customers are evaluating its use. The company has not identified those products or provided independent benchmark results for version 1.0. The software is available now, with further performance assessment possible using individual customer models and Apollo development boards.
