How many steps are there to install the large model into the phone?

At the beginning of the year, when ChatGPT became popular, I went to Shanghai to attend Huawei's Spring New Product Launch and mentioned the combination of large model technology and mobile hardware with a one-page keynote. Although it was only a short one or two minutes, I specifically posted a Moment to see the application of natural language interaction skills on mobile phones.

At that time, I had a premonition that combining the big language model with a mobile phone would be very fast.

Returning to the present, Apple, Samsung, and domestic smartphone manufacturers such as "Huami OV" have all announced the landing of large-scale models. For mobile phones, whether there is a big model is no longer a multiple-choice question, but a must answer question.

When I mention this story, I don't mean to say how accurate our predictions are. Observing technology is not metaphysical fortune telling, everything has rules to follow. 2023 is coming to an end, but the excitement of big model phones is still on the rise. It's time to summarize and discuss the deep logic behind the combination of phones and big models, what are the current differentiated gameplay and challenges for each company, and how they will develop in the future.

2023, the first year of infrastructure construction for mobile phone models

Firstly, it is necessary to explain why we predict that the combination of large models and mobile phones is inevitable?

Readers who are familiar with the recent situation of the mobile phone market should know that after more than a decade of glory, mobile smart terminals have fallen into a certain bottleneck, with sluggish growth and a lack of highlights. Manufacturer innovation is like squeezing toothpaste, and developers cannot cook without rice, with limited creative space available for implementation. At the same time, a user is surrounded by at least three or four mobile devices, and they have to put in a lot of invisible labor for tedious interactions every day. Some even get too tired and start trying "digital abstinence".

It can be said that both the supply and demand sides of mobile digital services are looking forward to a new transformation technology that can simplify the ecosystem of mobile intelligent terminals and reshape the experience. In today's technological world, large models are the best option.

The powerful understanding and generation capabilities of the large language model, with various functions obtained through the same model base and natural language interaction, can change multiple basic capabilities of the phone. It is not too much to redefine the phone.

Making big models run on mobile phones has become a must-have for mobile phone manufacturers, a window of opportunity for developers, and an inevitable option to reignite user enthusiasm.

Everyone has heard about the "Hundred Model Battle" this year, but unlike the general large language models that mainly train and call on the cloud, highly integrated mobile phones, a series of software and hardware such as end-to-end computing power, OS operating system, application concurrency, and UI interaction, all have their own characteristics, which also bring many limitations to the implementation of large models.

Based on the action plans of leading manufacturers such as Huami OV, we predict that 2023 will be the first year of infrastructure construction for the mobile phone model.

The large model landing phone, the infrastructure project has just begun, and there will be more exciting scenes to watch in the future. Specifically, manufacturers must address:

1. Three infrastructure projects.

2. An entrance.

3. A group of people.

Let's take a look at the current situation of each company from these three aspects? What are the common challenges and differentiated approaches?

The first step is to deploy end-to-end and build three infrastructure

The parameter quantity of a large language model is often tens or hundreds of billions, and its training and inference process require a large amount of computing resources. For mobile intelligent terminals such as mobile phones, the computing power of SoC chips is far from reaching the scale of a data center's 10000 card cluster. How can we support the end-to-end operations of the large model?

Even if I reluctantly bring it up, will the large model occupy too much of the phone's working memory and seize the resources of other applications, resulting in stuttering or rapid power outage?

Insufficient local computing, and the introduction of cloud computing will cause many problems, such as large models analyzing and processing personal data on the cloud, will it expose my privacy?

To deploy on the end side, both mobile phones and large models need to undergo some modifications.

Firstly, the model layer.

There are currently two main routes.

One is to make the large model smaller, which means introducing lightweight large models on the end side. By using compression techniques such as quantization, pruning, and distillation, the model structure and parameter size are adjusted to adapt to the memory and computing power characteristics of the end side chip, which can be used even without a network, represented by Honor and Xiaomi.

Honor Magic6 is equipped with a self-developed 7B side AI large model (i.e. 7 billion parameter scale). Lei Jun announced in his 2023 speech that "Xiaomi fully embraces large models", focusing on lightweight and local deployment. Currently, he has trained large models with 1.3B and 6B parameter scales.

The second is to make the large model long, providing large models with different parameter magnitudes to support different scenarios and tasks, and cloud collaboration, represented by vivo and OPPO.

The self-developed AI large model matrix released by vivo in November includes five large models with different parameter magnitudes of one billion, ten billion, and one hundred billion. Among them, the 1 billion level model is a professional text model mainly designed for end-to-end scenarios, the 7 billion level model is an end-to-end cloud dual-use model designed for mobile phones, and the 70 billion level model is the main model for cloud services.

Similarly, using matrix methods, there is OPPO's Andes Grand Model (Andes GPT), which includes models with different parameter scales ranging from 1 billion to 100 billion.

At the VDC conference, I experienced "vivo seeing" based on the Blue Heart model on site, which can provide real-time object recognition for visually impaired groups without any network. It can recognize plants, QR codes, bus cards, and other objects in a timely manner, and the heating and battery life of the phone are also within an acceptable range. It can indeed solve the actual needs of visually impaired groups to perceive the external environment when traveling outside.

This feature touched me a lot and I even posted a Moment to share it.

However, product personnel also bluntly stated that this large-scale model application with complete network disconnection and local computing has high performance requirements for mobile phone chips, and can only be implemented on some flagship models at present.

Secondly, the chip layer.

No matter how small a large model is, it is also a "big" model. blindly compressing it may reduce model performance and output quality, leading to a decrease in recognition accuracy and generated content. Therefore, the implementation of large models, hardware upgrades, especially mobile chips for mobile phones, are essential prerequisites.

According to the current information obtained, vivo, MediaTek, Qualcomm, and others have joint research and development collaborations to accelerate the optimization of AI inference performance on the mobile phone's end side. Xiaomi also revealed that they will work together with chip companies (Qualcomm and MediaTek) to promote the implementation of the end side large model. In addition, this year's Kirin chip returns and can achieve deep collaborative optimization with Huawei's Hongmeng operating system and Pangu Big Model.

It must be acknowledged that the application of large models has just begun, and the collaborative tuning with mobile chips has only taken the first step. In the future, mobile phones will carry video and image AIGC tasks, and the vast majority of users should not be willing to upload them to the cloud. Therefore, the optimization and tuning of local AI computing hardware will be one of the competitiveness of mobile phone manufacturers in the future.

Then, at the system layer.

Ultimately, efficient and usable large model applications must be collaborative across the cloud to balance experience and privacy. This has brought about some issues, such as how to ensure user privacy and data security in the cloud for data and business? Will AI applications based on large models affect phone performance, battery life, and user experience? To solve this problem, efforts must be made from the underlying operating system.

Among them, Huawei's flagship mobile phone is connected to the Pangu model through the HarmonyOS4 system, and the Hongmeng system is the OS written entirely as the underlying source code. With the addition of a microkernel architecture, the core operating system services and security services are separated, as well as security chips and isolation technology, providing a comprehensive security guarantee mechanism from both software and hardware.

In addition, OPPO's ColorOS, Xiaomi Pengpai OS, vivo Blue Heart Big Model, and its mobile phone system OriginOS4 have all become the ground for self-developed big models.

The ability of the large model to deeply integrate with the operating system, as well as the smoothness, security, and intelligence of the operating system itself, determine the key to the subsequent performance of the large model.

According to a staff member from vivo, in addition to the performance and quality of the base model itself, a large amount of engineering details are also essential. To enable the operating system to quickly execute user instructions, not only does the large model need to decompose the input voice/text through the thought chain, but it also needs to have a deep understanding of mobile skills, intelligently organize hundreds of skills, automatically select and call corresponding APIs, in order to automatically execute complex tasks, leave complexity to oneself, and leave simplicity to the user.

It is not difficult to see that the underlying model, chip, and operating system are the three essential infrastructure for the end-to-end deployment of large models. At the same time, it should also be noted that the next stage of the mobile phone market is high-tech, difficult, high investment, and high-risk, requiring solid "hard work". Competition will become severe, and players will also become fewer.

Step 2: Touch the user and create an entry point

The big model has been popular for a whole year, and the aspirations of big manufacturers, media, and entrepreneurs have surpassed ChatGPT and benchmark GPT4, but to ordinary readers, it seems unclear: how can the big model change my life?

So, how about turning smartphones into "Aladdin's Lamp"?

Readers who have read fairy tales know that the universe in the lamp is vast, containing countless resources and treasures. However, Aladdin does not need to worry about it. He only needs to express his wishes, and there is a "lamp god" who arranges everything for him. The ability to understand and create large models empowers mobile voice assistants, turning them into "lamp gods".

The voice assistant with the ability to access large models is a direct path for mobile phone manufacturers to reach users.

At present, when it comes to mobile phones, big models can do three things: first, utilize the natural dialogue ability of big language models to change the terminal interaction experience; Secondly, utilizing the understanding ability of large models to provide personalized services, familiarizing users with their daily preferences and habits, and gaining a better understanding of users; Thirdly, utilizing the creative power of large models for abstract extraction, copy generation, and image production to improve productivity

And the above abilities are basically achieved through a voice assistant step by step.

For example, Huawei's smart assistant Xiaoyi has been enhanced in three directions: intelligent interaction, efficient productivity improvement, and personalized services by integrating into the underlying capabilities of the Pangu model.

The vivo Blue Heart model is combined with the mobile system OriginOS4 to create the first global intelligent assistance "Blue Heart Mini V", which can help users complete many complex tasks through natural communication and simplify complexity.

The new Xiaobu assistant based on the OPPO Andean model, as well as Xiaoai classmate who upgraded the Xiaomi AI model, have also been launched in beta and experiential versions.

In the era of the Internet of Things, intelligent end users will face a big explosion in devices, information, and services. If everything requires users to search for it themselves, it is like looking for a needle in a haystack. This places high demands on everyone's patience, time, and digital information literacy. The fusion of big models and intelligent assistants is the antidote.

Intelligent assistants can schedule IoT devices such as mobile phones, headphones, cars, tablets, smart screens, computer PCs, smart homes, etc. They are the shortest path between users and AIoT.

With the support of large models, the analytical and understanding abilities, knowledge levels, memory levels, and generative abilities of intelligent assistants have greatly improved, making the "thousands of people and machines" that mobile phone manufacturers have been saying for many years truly achievable and perceptible.

But for a long time, users did not feel that intelligent assistants were indispensable. Some people also found it awkward to talk to intelligent assistants, have artificial intelligence impairments, and even feel a bit weak.

It is not an exaggeration to say that ordinary users may not necessarily care whether the large model is easy to use, but whether the intelligent assistant is easy to use can be tested in a single conversation, which will be the focus of competition for mobile phone manufacturers in the future.

To summarize, having a large model alone is not enough. Intelligent assistants are the shortest path to reach users and a must compete ground for mobile phone manufacturers.

Step 3: Application prosperity, a group of developers

After mobile phones and users have met the prerequisites for accessing the big model, the next step is how to truly enable developers to enter the world of big models.

Today, at the press conferences of manufacturers such as Huawei, vivo, and OPPO, I have seen many AI applications based on large models, but they tend to be demonstration oriented, such as natural language intelligent search, one sentence image generation, AI composition, and so on. These functions are still too basic for general users.

Taking the popular AIGC application as an example, generating the most beautiful ID photos, drawing pets, creating manga avatars, writing a sweet potato sharing copy, Saibo Bodhisattva, taking pictures to do math problems, and generating smart watch wallpapers are all highly segmented in demand. Mobile phone manufacturers cannot, and should not, do all of these AI applications themselves, which requires the introduction of millions of developers to unleash creativity and create countless small and beautiful AI applications based on large models.

However, from the development end to the market end, the path of large model AI applications may seem tempting, but for developers, they still face various concerns such as technology, learning costs, and market pressure, requiring strong technical systems, tool platforms, empowerment solutions, and the support of business potential from manufacturers.

Currently, we can see several ecological strategies:

Hongmeng's technological path.The market space for full scene intelligence and the industrial capacity of Hongmeng distributed system are still very attractive to developers. Huawei is ready to launch the new HarmonyOSNEXT and fully launch Hongmeng native applications.

Vivo's Open Source Path.Vivo has embarked on a path of open source co construction, with the 7 billion yuan Blue Heart model becoming the first Chinese open source model in the industry. The benefits of open source are that it can attract group wisdom, making it more suitable for extensive and borderless exploration during the technology exploration period, thereby generating more updated and better creative applications. Vivo has also released corresponding fine-tuning frameworks and the large model development kit BlueKit, providing comprehensive support for developers.

OPPO's path of partnership.Previously, OPPO announced the content outlook for the 2023 OPPO Developer Conference, in which the Pantanal system's capabilities were fully open to developers, supporting rapid adaptation of multiple forms, modalities, and entrances, and providing corresponding tool resources to help developers quickly access ubiquitous services and attract more partners to provide diversified intelligent services.

It can be confirmed that with the increasing number of big models, the gradual maturity of basic software and hardware for mobile phones, and the improvement of platform capabilities and tool interfaces, mobile AI applications based on big models will become more and more popular. At this time, only the quantity and quality of developers can widen the gap in the application ecosystem - how much exciting imagination developers can unleash, the value of mobile phones will be.

The developer ecosystem is the moat of the mobile internet era, and this principle also applies in the AI big model era.

For mobile phone manufacturers, fortunately, large model phones are just beginning, and application developers do not want to miss the opportunity window and have time to accumulate chips.

To summarize, 2023 is the year of cultivation for large model smartphones, with three infrastructure buildings, one entrance, and a group of people gradually converging on the end, and changes may be occurring in an instant.

When the killer app of the big model steps into the phone, it makes users scream "wow". The "aha moment" of this large model phone indicates that the next spring of mobile internet has truly arrived.