Inertial Navigation on MCU
Write a GNSS/IMU/wheel dead-reckoning engine in C for an emulated Cortex-M4F, graded on 39 held-out real drives and metered CPU/RAM/flash budgets.
Results
Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.
| # | Model | Rollouts | Pass@1 | Score | Time |
|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 max | 4/4 | 2.1h | ||
| 2 | GPT-6.1 Sol max | 4/4 | 5.3h | ||
| 3 | Claude Sonnet 5.5 max | 4/4 | 7.2h | ||
| 4 | GPT-6 Astra xhigh | 3/4 | 6.0h | ||
| 5 | GPT-5.6 Sol max | 0/4 | 4.5h | ||
| 6 | Gemini 3.8 Flash high | 0/4 | 2.2h | ||
| 7 | Qwen3.8 Max xhigh | 0/4 | 8.2h | ||
| 8 | GLM 5.3 Flash max | 0/4 | 9.7h | ||
| 9 | Muse Spark 1.3 max | 0/4 | 8.6h | ||
| 10 | Kimi K3 max | 0/4 | 10.3h | ||
| 11 | Grok 4.7 xhigh | 0/4 | 7.4h |
Background
A fleet telematics box must report vehicle position when GNSS (satellite navigation) drops out or reflects under bridges and in tunnels. The answer is an inertial navigation engine fusing GNSS with a MEMS IMU (accelerometers and gyroscopes) and wheel ticks on a microcontroller with fixed CPU, RAM and flash. Here the engine runs on a 64 MHz Cortex-M4F with real IMU and wheel recordings from one passenger car in southern Ontario, across seasons. The sensors are realistically bad: an IMU mount known only to within 6 degrees per axis, drifting MEMS errors, a wheel-tick scale off by up to 3%, and GNSS with latency, outages and multipath biases up to 25 m.
Task
The agent delivers causal C source implementing a fixed header, with one entry point per IMU sample, wheel message and GNSS fix and a 10 Hz output of position, velocity and heading. It is compiled with fixed flags, linked with a firmware harness and run on an emulated board with no heap. Every executed instruction is charged from a published cost table. Per-call limits are 14,000 units for an IMU sample, 50,000 for a wheel or GNSS message and 2,000 for an output. The mean budget is 1.6 million units per second of drive, with 2 KiB of stack, 6 KiB of static RAM and 40,000 bytes of flash.
Workspace
The offline sandbox (4 CPUs, 8 GB, 12 hours) holds 13 visible drives of 9 to 51 minutes, each with input events, a survey-grade reference at every output time, and metadata listing outages, multipath episodes and tolerances. Tools build as graded, run drives on the metered emulator or natively, and create worst-case drives. The 39 graded drives are held out.
Verification
The source is rebuilt and run on 39 held-out drives of 9 to 60 minutes: other recordings of the same car, some on unseen roads, about a third stress drives at the stated bounds. Every drive must process every event within budget, without faults. Four metrics per drive: 95th-percentile horizontal error outside outages, worst error inside outages of 20+ seconds, 95th-percentile multipath error, and 95th-percentile heading error. Each tolerance is 1.3x the worse 99th percentile of two independent reference engines over 100 noise realizations. A pass requires 36 of 39 drives within tolerance on every metric, with at most one metric beyond twice its tolerance.
Environment
- Toolchain
- arm-none-eabi-gcc 12.2 (newlib), QEMU 10.1 (mps2-an386) with metering plugin, Python 3.12 with NumPy
- Compute
- 4 CPUs · 8 GB RAM
- Network
- Offline
- Time limit
- 12 hours
References
- Boreas dataset: Source of the real IMU and wheel recordings and the post-processed survey reference; GNSS fixes and sensor errors are generated on top.
- OpenStreetMap: Bridges, tunnels and covered ways along each route determine where GNSS fixes are withheld.
- QEMU: Emulates the Cortex-M4F board, with a plugin that meters every executed instruction against the cost table.