ByteDance's 'Doubao' Mobile Assistant Preview

ByteDance has released a technical preview of the Doubao mobile assistant, showcasing various capabilities both on-device and in the cloud. - Supports power button, voice, and headphone wake-up. - Enables immersive real-time calls (including video) and voice-based photo editing. - Allows cross-app operations, simulating clicks and inputs to complete complex tasks (App Agent/Phone Operation). - Features on-device memory, capable of remembering images, texts, and schedules to proactively assist users (e.g., finding parking spots, retrieval codes, train seat bookings). - Privacy statement: emphasizes local processing and storage, allowing users to turn memory features on or off at any time. - Pro mode integrates GUI Agent with API calls to handle complex long-chain tasks (e.g., planning a trip to Paris and booking tickets across platforms). - Multimodal generation and editing are integrated into system-native applications, such as intelligent photo editing with voice commands within the photo gallery.
Key insights
Multimodal and Cross-App Operations as Core Differentiators of Doubao
The technical preview of the Doubao mobile assistant demonstrates its cross-app operation capabilities: AI can simulate clicks, inputs, and swipes to automatically execute multi-step tasks (e.g., price comparison across the web, service reservations, office workflow automation). Additionally, multimodal generation/editing capabilities are embedded in system-native applications, supporting intelligent photo editing via voice commands in the photo gallery, showcasing the differentiation in system-level integration and automated execution.
On-Device Memory and Privacy Control Coexist
Doubao features on-device memory functionality, capable of extracting key information from historical conversations and screen data for subsequent tasks (e.g., answering 'Where is my car parked?' based on a photo or checking seat numbers from ticket purchase records). At the same time, it announces privacy protection measures, emphasizing local data processing and storage, and allowing users to turn memory features on or off at any time.
Pro Mode for Complex Task Orchestration and Execution
Doubao's Pro mode combines GUI Agent with external API calls, providing enhanced reasoning and decomposition capabilities for complex and ambiguous long-chain tasks. Example process: interpreting vague user commands -> retrieving preferences from memory -> filtering targets -> cross-platform ticket booking and mapping -> generating reminders, showcasing the execution link between on-device and cloud collaboration.