The consumer version of Doubao Mobile Assistant is officially released, first installed on Nubia mobile phones

On September 14, the consumer version of Doubao Mobile Assistant was officially released.

Doubao Mobile Assistant is based on Doubao App and cooperates with mobile phone manufacturers at the system level to further explore the application of AI in mobile phones around core experiences such as interaction, task execution, local memory and retrieval. According to the official video, compared to the technology preview version launched at the end of last year, this consumer version emphasizes stability and daily usability.

Interactive experience is one of the basic capabilities highlighted in the consumer version of Doubao Mobile Assistant. According to reports, Doubao Mobile Assistant supports multiple wake-up methods such as voice and AI keys to adapt to different usage scenarios. Among them, the exclusive AI key has fingerprint authentication capabilities. After completing personal verification, you can call the assistant to perform related tasks without unlocking it twice. In terms of voice wake-up, Doubao Mobile Assistant is also optimized for far-field, internal noise and noisy environments.

In addition, it is worth mentioning the on-screen question and answer capability of the mobile assistant. Users do not need to take additional screenshots or switch apps, and the assistant can continue processing tasks based on the current screen content. For example, in the official video demonstration, the user turns on the camera, points it at the indoor environment, and says, "Based on this decoration style, help me choose a cabinet within 1.2 meters and priced at no more than 1,000 yuan." Doubao can combine the screen information and user requirements to find qualified product recommendations on the e-commerce platform.

Similar screen context understanding capabilities have previously appeared in Apple's demonstrations of Siri and Apple Intelligence. This means that mobile phone AI assistants are gradually shifting from simply answering questions to understanding the specific scenarios and actual needs of users, and further completing corresponding tasks.

In addition to interaction capabilities, personal context and memory capabilities are also key to whether mobile assistants can meet users' personalized needs. According to reports, Doubao Mobile Assistant provides a search tool for local data, which can retrieve photo albums, text messages, notes and other information when users need it. For example, when a user asks about the expiration date of a Hong Kong and Macau pass, the assistant can find the answer from the relevant information on the phone.

Focusing on office scenarios, the consumer version has also upgraded the recording experience and has access to Feishu Miaoji capabilities. Combined with local search tools, users can directly ask for specific content, such as "product recommendations mentioned in yesterday's user interview." In addition, users can also manually save the current screen content through voice input, three-finger swipe up, or hold down the AI ​​key and volume key at the same time to form a memory that can be retrieved locally.

Task execution is another core capability demonstrated by Doubao Mobile Assistant this time. Taking the travel scenario as an example, if the user says "call me a car to go to the company", Doubao can confirm the usual address, select the car model and initiate the order.

It is worth mentioning that the "operating mobile phone" function, which has received widespread attention after the release of the technical preview version, is now open in Beta version. Doubao said that graphical interface intelligence is still a rapidly developing cutting-edge capability, and Doubao Mobile Assistant hopes to work with partners to build an open and standardized mobile AI application ecosystem. To this end, Doubao Mobile Assistant launched the Screen Automation Execution Protocol (SAEP) and simultaneously launched a 30-day rule announcement. According to this agreement, third-party applications can independently declare operating boundaries and explicitly allow or deny AI assistants to perform screen automation operations within this application. Doubao Mobile Assistant stated that it will fully respect the wishes of developers and will not perform automated operations on third-party applications that explicitly refuse relevant operations.

At the same time, Doubao Mobile Assistant also focused on privacy and security related work in the released video. According to reports, Doubao Mobile Assistant follows the principles of "reasonable necessity" and "user independent control" in data collection and processing. At the same time, it has established an Agent protection system to conduct hierarchical and hierarchical management of the operable scope. The release video also mentioned that users can go to the official website of Doubao Mobile Assistant to view the complete privacy and security white paper.

It is reported that the first consumer version of Beanbao Mobile Assistant will be installed on the Nubia NaviX Ultra mobile phone. Pre-orders are currently open and will be officially released on September 16.