Robot AI · October 3, 2026
The power budget of on-device AI in robots: ask about watts before compute
How much AI fits in a robot is usually decided by the battery, the cooling and the enclosure. Here are five questions engineering teams should answer first when evaluating on-device large models.
In a robot, compute is only part of the power
A mobile robot's power is shared by drive motors, sensors, communications, the main controller and AI compute. With a fixed battery, every extra watt spent on AI shortens runtime; the more sealed the enclosure, the less heat it can dissipate. For robots, the limit on AI capability is often not how fast a chip can run, but how many watts the machine can give it.
Why peak compute is a poor metric
Peak compute describes performance under ideal conditions. But when a large model generates output, it is mostly limited by data movement, not arithmetic. Two chips with the same peak compute can differ widely in real token rate and in energy per task if their memory architectures differ.
The more useful questions are: with your model and your power limit, how many tokens per second can you generate, and how much energy does each task take?
A starting point: one simple formula
When weights sit in external memory and requests are processed one at a time, you can estimate the speed ceiling with:
- token-rate ceiling ≈ external memory bandwidth ÷ bytes read per token
The power spent on data movement grows roughly with bytes moved per token × token rate. That is why reducing external traffic per token improves both speed and power.
Five questions to answer first
- What power and thermal budget does the robot leave for AI — sustained, or short peaks?
- What are the target model's parameter count, quantization bit-width and context length?
- Which decisions must be made locally, and how much latency is acceptable?
- Can site data leave the device or the premises?
- How should AI compute be split with the existing controller, and what interface and OS constraints apply?
The co-processor approach
Many robots already have a proven controller and motion-control software, and redesigning the controller is costly and risky. Handing large-model inference to a dedicated co-processor, while the controller keeps motion control and safety, is a lower-impact path. VXI-IR5 is an on-device large-model co-processor designed around exactly this idea.
To discuss your robot and power constraints, see Robotics & Embodied AI, or start with the model memory estimator.