World models & multimodal prediction advances

World-model research and native multimodal predictors advance rapidly.
Key insights
Yann LeCun leaves Meta to found AMI and champion world models
Yann LeCun left Meta at age 65 to found Advanced Machine Intelligence (AMI) in Paris, citing Meta's growing closedness and short‑term focus. He criticized the LLM‑only path, arguing LLMs trained on text cannot reach human‑level real‑world intelligence because text lacks necessary spatio‑temporal data. LeCun advocates world models implemented via Joint Embedding Predictive Architectures (JEPA/I‑JEPA) and related methods (Barlow Twins, VICReg), and commits AMI to publish upstream research while also building products.
Emu3.5 unifies multimodal inputs to predict next world state
Beijing institute released Emu3.5, a multimodal world model that tokenizes image, text, and video into a unified token stream and trains by predicting the next token/state (NSP). Emu3.5 reframes tasks so next‑token prediction corresponds to predicting the next visual or linguistic world state, enabling spatial reasoning, temporal dynamics, and cross‑modal continuity within a single model.
World models vs. pixel/text prediction: JEPA and representation prediction
LeCun and collaborators argue that world models should predict in an abstract representation space rather than at pixel/token level. JEPA-style approaches encode X and Y into embeddings and predict future embeddings, which filters out unpredictable low‑level noise. Developments cited include Barlow Twins, VICReg, SigReg and I‑JEPA variants ensuring isotropic encoder outputs; these techniques were refined over decades and applied recently to video and multimodal prediction tasks.
Native multimodal transformers improve rendered text and instruction following
Native multimodal token‑in/token‑out models such as Nano Banana Pro and gpt‑image‑1 treat image pixels and text as shared vectors; this allows the model to 'write' readable Chinese characters inside images rather than approximating strokes as pixels. The result is much lower character error rates and improved instruction following in images compared with diffusion‑based pixel models.
DeepSeek‑V3.2: sparse attention and expert distillation for efficient reasoning
DeepSeek‑V3.2 introduces a DeepSeek Sparse Attention (DSA) mechanism to reduce complexity on long inputs, trains via specialist expert models plus knowledge distillation, and uses large synthetic simulated task suites (≈1800 scenarios) for agent training. Paper reports parity or superiority to closed models on reasoning and agent benchmarks while lowering compute costs.
Runway releases Gen 4.5 and GWM1 world models for video and environment simulation
Runway published Gen 4.5 video models with original audio generation/editing and ALF for multi‑shot video editing keeping character and lighting consistency. Runway also released GWM1, an autoregressive world model for per‑frame prediction that supports streaming generation with user intervention, plus GWM Worlds for real‑time environment simulation, GWM Avatars for high‑fidelity audio‑driven video generation, and GWM Robotics for simulating successes and failures for robot policy evaluation.