XNews

Multimodal world models, video and avatar advances

Multimodal world models, video and avatar advances

Multiple multimodal and world-model advances were published across organizations. - Beijing institute released Emu3.5, a unified multimodal world model that tokenizes image/video/text and predicts 'next world state' - Nano Banana Pro and native multimodal models improved image text rendering by treating text as cross-modal tokens - Runway announced Gen 4.5 video models with original audio generation/editing, ALF multi-shot video editing, GWM1 autoregressive frame-predictive world model, GWM Worlds environment simulator, GWM Avatars high-fidelity digital humans and GWM Robotics for sim-driven policy evaluation - Qwen3-Omni claims end-to-end omni-modal unification (text, image, audio, video) with SOTA across audio/AV benchmarks and open-sourced variants - Improvements emphasize coherent multi-frame consistency, audio-driven avatar motion, and world-model predictions for next-frame realism

Key insights

Emu3.5 proposes unified tokenized world modeling across modalities

A Beijing research institute introduced Emu3.5, a multimodal world model that converts images, text and video into unified tokens and treats learning as next-state prediction. Emu3.5 frames tasks as predicting the next world token (text, next frame, next event), claiming better spatio-temporal and causal consistency across modalities and enabling next-step forecasting for visual and linguistic sequences.

Runway releases Gen 4.5, GWM1 and avatars for video generation and editing

Runway released Gen 4.5 video models with native audio generation and editing, ALF for multi-shot editing with environment consistency, and GWM1, a generative world model for autoregressive frame prediction. Runway also published GWM Worlds for immersive environment generation, GWM Avatars for high-fidelity audio-driven digital humans, and GWM Robotics for simulating both successes and failures to support robot learning and strategy evaluation.

Native multimodal models improve rendered text and instruction following

Native multimodal transformers like Nano Banana Pro and GPT-image-1 achieve markedly better in-image text rendering and instruction following by treating image elements and text as shared tokens, reducing spelling errors and improving cross-modal fidelity versus diffusion-based pixel models.

Qwen3-Omni promotes end-to-end omni-modal capabilities and open-sourcing

Alibaba’s Qwen3-Omni claims unified end-to-end handling of text, image, audio and video with SOTA results on many audio/AV benchmarks, low latency and open-sourced variants for instruction, thinking and captioning. The release includes public weights and developer tooling for experimentation.

Source posts