RoboBuilderAgentic Robot Co-Design
with Real-world Video Feedback in the Loop

Ghadi Nehme Dahyun Nam Akshay Govind Srinivasan Faez Ahmed

Department of Mechanical Engineering, Massachusetts Institute of Technology

Can AI agents design robots that work in the real world?

16,427simulations
12,774robots designed
135robots built by hand
66%of simulated passes hold for real
72%of level-1/2 real failures removed by the loop

Designed in simulation by a coding agent, then built and tested on the real course. Inset: the simulation. Tap to swap.

98%
of level-1 tasks solved by agents. Search baselines: 0%.

01Benchmark

Ten obstacles, three levels, one parts bin. Each agent designs a robot and its open-loop motor program; the robot must reach the goal in time and stay upright for 5 s.

132part types, ~3,000 in stock, motors included
+1custom part the agent writes as CAD code; we 3D-print it
114pieces in the median robot that passes
Cross30 sims +Print+30 sims Light50 sims Fast50 sims All-in-one1 robot, 10 courses

Leaderboard

02The loop

Every robot that passes in simulation is built and tested on the real course. Every real failure goes back to the agent as a video.

0 / 15early sim passes that crossed for real, with public part data
4.5 N·mmmeasured friction-pin hold, not the assumed 24.5
92%of labelled real runs predicted once the agent learned the simulator from video
≤ 2parameters an agent may change per failure, kept only if the old design then fails as filmed

03Real world

Two in three simulated passes also cross the real course. Shown a video of its failure, the agent fixes most of the rest.

1Built from the simulated design

135robots built by hand
~13kpieces
~240 hestimated building and testing
461real videos, each paired with its simulation

2How the rest fail

3The agent watches its failure and redesigns

72%of level-1/2 real failures removed in one round

04All-in-one

One robot for all ten obstacles crosses at most six of them. Bumps want clearance, the bridge a narrow track, the rope a light chassis hung from a cord. No single robot does it all.

05Explorer

Thousands of agent-designed robots on one map. Each dot is an agent design, placed by how it looks. Ringed dots were built: open one to watch it in simulation and on the real course.

Hover or tap a dot · drag to pan · click the map, then scroll to zoom · pinch on touch

Obstacle
Level
Agent
Show

BibTeX

@article{nehme2026robobuilder,
  title={RoboBuilder: Agentic Robot Co-Design with Real-world Video Feedback in the Loop},
  author={Nehme, Ghadi and Nam, Dahyun and Srinivasan, Akshay Govind and Ahmed, Faez},
  journal={arXiv preprint arXiv:2611.23855},
  year={2026}
}

Scope. The simulator is fitted from a fixed camera and bench tests, so not every parameter is identifiable; each loop correction is fitted to one failure; Light and Fast designs were not built. We release the simulator, courses, designs, simulations and 461 paired real videos.