RoboColosseum Launches Open Real-World Robot Evaluation Initiative

RoboColosseum launched an open robot evaluation initiative with 100+ robots across 140+ in-the-wild scenes, offering 10,000 free evaluation hours to academic labs. The platform evaluated five VLAs and three frontier models across 20,000+ runs, with policy rankings varying by task and robot. All rollout data, including failures, is open-sourced.
Event version 1.
Key insights
Large-Scale Real-World Fleet
RoboColosseum deployed one of the largest dedicated robot fleets across 140+ scenes and thousands of in-the-wild tasks. The fleet includes Unitree G1s, bimanual i2rt YAMs, single-arm LeRobot SO-101 and Franka robots, and frodobots Earth Rovers.
Two Evaluation Tracks
The platform has two evaluation tracks: open-world zero-shot performance in blind tests and fine-tune within-distribution performance after fine-tuning on fixed, open-source datasets. Across 20,000+ runs, policy rankings varied by task and robot.
Models Evaluated
Five VLAs were evaluated: MolmoAct2, GR00T N1.7, π0.5, LingBot-VLA v2, and G-0.5. Three frontier models were also tested: GPT-6 Astra, Opus 5.5, and Grok 4.7, with evaluators manually judging results across different robot embodiments.
Open Data and Free Evaluation Hours
All rollout data from the evaluations, including failures, is open-sourced on Hugging Face and LeRobot. A shared pool of 10,000 free evaluation hours is available to academic labs.
Founding Partners
RoboColosseum is led by four founding partners: Hugging Face and LeRobot host rollout data and model checkpoints, BitRobotNetwork runs real-world deployment operations, eaic_sg provides access to in-the-wild environments, and DJiafei and MAGIC Lab at NUS set research direction.