Robotics and world models: Gemini Robotics & Emu3.5

DeepMind and Beijing groups presented progress on robot cognition, world models and multimodal prediction. - DeepMind's lead on robotics explained gap between simulation success and real‑world robot reliability. - Gemini Robotics 1.5 adds textual 'thinking' outputs and motion‑transfer to enable cross‑robot skill sharing. - Motion Transfer enables data from one robot morphology to help others, addressing data scarcity. - DeepMind stresses data cost and advocates scalable simulated and generated data sources (YouTube, generated video) for training. - Tactile sensing gains importance for dexterous hand tasks; vision may suffice for coarse grasping but not fine manipulation. - Timeline estimates: 2–3 years for noticeable breakthroughs, 5 years for vertical deployments, 10 years for wider household presence. - Emu3.5 (Beijing) is a unified multimodal world model that tokenizes image/text/video and trains to predict the 'next world state' (NSP), enabling next‑frame/world predictions across modalities. - Emu3.5 reframes generation as next‑token world prediction rather than modality‑specific outputs.
Key insights
Gemini Robotics 1.5 adds explicit 'thinking' and cross‑morphology transfer
DeepMind's robotics work embeds a textual planning/’thinking’ stage before motor output so robots produce human‑readable intermediate reasoning. Gemini Robotics 1.5 includes motion transfer techniques that allow data from one robot body to inform another, reducing per‑platform data requirements. The team highlighted that adding readable internal steps improves multi‑step task handling and transparency. (supporting_posts: [16783])
Data scarcity remains the central bottleneck for real‑world robotics
DeepMind emphasized that robot data collection is expensive and not scalable at present; estimates include large hardware, human teleop and maintenance costs. The team argues for using scalable sources—simulation, internet video and generated data—to close the gap. Motion transfer and simulation fidelity are described as core strategies to make data practical for broad robotic generalization. (supporting_posts: [16783])
World models shift to unified token prediction for multimodal continuity
Emu3.5 presents a unified multimodal approach that converts images, text and video into tokens and trains the model to perform next‑state prediction (NSP). Emu3.5 treats the prediction of the next token as prediction of the next world state (visual or textual), enabling cross‑modal temporal reasoning and prediction. The architecture aims to learn physics, causality and temporal evolution across modalities. (supporting_posts: [16324])
Tactile sensing grows in importance for dexterous manipulation
DeepMind noted that while vision suffices for many grasping tasks, fine manipulation with dexterous hands (e.g., scissors, small objects) requires tactile feedback. The team observed that touch becomes necessary as robotic manipulators advance from simple grippers to multi‑finger dexterity. (supporting_posts: [16783])
Developers expect staged real‑world impact over decade timescales
Interviewees provided timelines: 2–3 years for perceptible breakthroughs, 5 years for vertical deployments in logistics/manufacturing/retail, and ~10 years for broad household adoption. They cautioned that public demos often show cherry‑picked successes; general reliability remains below production thresholds. (supporting_posts: [16783])