Vision-Language-Action inference, measured
How fast does your VLA run on our engine?
Search measured latency and memory for 16 open VLA policies on 16 devices, from an RTX 5090 to a Raspberry Pi 5. Two complementary engines produced every number: vla.cpp for GPUs, NPUs and CPUs, and vla.simd for CPU-only robots.
- Measurements
- Models
- 16
- Devices
- 16
- Hardware targets
// explorer
Every measurement, searchable
Filter by engine, processor, model or device. Each row is one call to the policy: one action chunk from the camera frames and instruction. Open a row for its full statistics, memory, checkpoint and source report.
| Model · engine | Hardware | Setup | Latency, ms per call, log scale | Actions / s | Memory |
|---|
No measurement matches. Try fewer words or clear a filter. If we haven't measured it yet, ask for it or send us your numbers.
// engines
Two engines, one map of VLA inference
vla.cpp brings a wide set of VLAs to every accelerator through ggml. vla.simd goes deep on the CPU with its own kernels and an INT8 path, down to a Raspberry Pi 5. Where one stops, the other starts.
Where each engine runs
Models measured on each class of hardware. An empty cell means no report for that pair yet.
Reach for vla.cpp when…
- the robot has an accelerator: an NVIDIA GPU or Jetson, Apple Silicon, an Intel GPU or NPU, or a Snapdragon NPU;
- you need a large VLA such as π0, π0.5, GR00T or OpenVLA-OFT;
- you want one binary and one GGUF per model across all of them.
Reach for vla.simd when…
- the robot has only a CPU, from a Raspberry Pi 5 to a desktop;
- your policy is ACT, IMPACT, SmolVLA, Octo-Small, TurboVLA or Diffusion Policy;
- you can trade some numerical precision for INT8 speed, or serve LeRobot's async client.
// compare
Same policy, same CPU, two engines
Three policies run in both engines. On the CPUs that both measured, this is how their latency lines up. Lower is better.
- The engines time different checkpoints: vla.cpp the LIBERO fine-tunes, vla.simd the SO-101 policies, except TurboVLA, which is LIBERO in both.
- vla.cpp reports the mean of 20 calls with a synthetic 16-token prompt; vla.simd the median of 50 calls, encoding the instruction on every SmolVLA and TurboVLA call.
- Each engine runs its own fastest thread count and flags for that CPU, as listed in the row's details in the explorer.
- On the Core Ultra X7 358H, vla.cpp ran in the balanced platform profile (25 W long-term limit) and vla.simd in the performance profile.
// models
The policies
Every checkpoint is a single GGUF on Hugging Face. Pick a policy to see it on every device.
// method
How the numbers were measured
What one number means
Latency is the wall time of one predict call in the engine's own process: image preprocessing, every encoder, all denoising steps and action decoding. It excludes the network, the simulator and the robot, and it says nothing about task success.
Actions / s is the action chunk divided by the latency: how many actions a policy can supply per second when the robot plays each chunk in full. At 30 actions/s or more it keeps up with a 30 Hz control loop.
By default the explorer shows the fastest configuration each report found for a model on a device and processor. Flash attention, lower-precision weights and INT8 can move the predicted actions, so a checkpoint's success rate at the defaults may not carry over. Tick Show every config to see the defaults as well.
vla.cpp
vla-bench on synthetic inputs, at each model's native views and input size: 3 warmups and 20 timed calls per process, 3 processes, and the process with the lowest mean is reported. Memory is the peak VRAM and host RSS, or the RSS alone on unified memory. Every device ran commit , except the Core Ultra X7 358H, at 450992c (one OpenVINO fix later).
Two rows differ: the RTX 5090 is its PR author's minimum over 5 rounds, and the Apple M5 Max comes from an earlier build. Task success is from LIBERO on an RTX 3090, 200 episodes per suite.
vla.simd
vla-simd-serve --bench with fresh random frames every call: 5 warmups and 50 timed calls (20 when a call takes more than 3 s), and the median is reported. Each model runs at its fastest measured thread count and settings, in FP32 and in INT8 (W8A8) where the CPU has AVX-VNNI or Arm dotprod. Memory is the peak process RSS.
Every device ran commit , except the Core Ultra X7 358H, at c60b286 (the same code, merged into main). The reports measure no accuracy; INT8 changes the numerics.