XNews

Google / DeepMind Gemini real-time multimodal APIs and features

Google / DeepMind Gemini real-time multimodal APIs and features

Google/DeepMind announced Gemini Live API and related multimodal features for real-time audio, TTS control, and developer agents. - Gemini Live API (Gemini 2.5 Flash Native Audio) provides native audio I/O with low latency. - Live API replaces STT->LLM->TTS chains with single WebSocket streaming. - Features: native emotional tone detection, smart interruption handling, tool calls, persistent memory, multi-region enterprise support. - Google Translate integrated Gemini for real-time speech-to-speech translation; beta supports 70+ languages and regions (US, Mexico, India) on Android. - Gemini TTS enables fine-grained control of gender, pitch, age, pronunciation via prompts. - Nano Banana powers a virtual try-on feature in Google Shopping using a single selfie and auto full-body completion. - Deep Research Agent (Gemini Deep Research) and Interactions API released for developers with templates and demos; agent achieved SOTA on internal research benchmarks.

Key insights

Gemini Live API provides native, low-latency multimodal streaming

Google released Gemini Live API based on Gemini 2.5 Flash Native Audio, enabling single-WebSocket connectivity that ingests raw audio and emits audio responses without separate STT/TTS steps. The API supports low-latency, multi-modal fusion (audio, video, text), tool calls during voice sessions, emotion and interruption handling, persistent context/memory, and enterprise-grade regional availability. Google supplied quickstart templates (Vanilla JS, React) and demos for advisor, multimodal customer support, and game assistants.

Google Translate integrates Gemini for real-time speech translation

Google integrated Gemini into Google Translate for real-time speech-to-speech translation with two modes: Continuous Listening and Two-way Conversation. The beta in Google Translate Android supports over 70 languages in selected regions (US, Mexico, India) and works with any headphones; iOS and wider rollout are scheduled for 2026.

Gemini TTS offers prompt-driven, fine-grained voice control

Google introduced a new Gemini TTS model that allows developers and creators to control speaker attributes (gender, tone, age), specific word pronunciations, and style via prompt instructions without switching voice models. The capability was highlighted by developers building interactive dramas combining Nano Banana Pro visuals with TTS-driven character voices.

Gemini Deep Research Agent and Interactions API target rigorous developer research workflows

DeepMind released Gemini Deep Research Agent—an agent trained for autonomous, multi-step research planning and evidence-backed reporting—built on Gemini 3 Pro. Google also launched the Interactions API as a unified interface to manage both models and agents; the agent achieved top internal scores on DeepSearchQA, Humanity's Last Exam, and BrowseComp benchmarks. Developers can integrate via GoogleAIStudio and use prompt-defined output structures, including JSON and tables.

Nano Banana virtual try-on adds single-selfie full-body completion

Google introduced a virtual try-on feature powered by Nano Banana models that generates a realistic digital avatar from a single selfie and auto-completes the user's full-body image; the feature lets users virtually try clothing on Google Shopping with photorealistic lighting and studio-quality appearance without requiring a full-body photo.

Developer tooling and templates accelerate production multimodal apps

Google provided developer templates (Vanilla JS, React) and three demo scenarios (real-time consultant, multimodal customer support, game voice assistant) to accelerate adoption of the Live API and Interactions API; the combined tooling aims to reduce integration latency and complexity for production voice and multimodal experiences.

Source posts