Dialogue with Zhou Hongyi: Making a bigger model is a long-lasting battle

Zhou Hongyi never conceals his optimism about the development prospects of big models. He, who claims to be a GPT evangelist, has been making frequent statements since the outbreak of ChatGPT

Zhou Hongyi never conceals his optimism about the development prospects of big models. He, who claims to be a GPT evangelist, has been making frequent statements since the outbreak of ChatGPT.

In his view, ChatGPT represents an important milestone in artificial intelligence, a singularity in general artificial intelligence, and a turning point in the arrival of strong artificial intelligence.

The above is just the beginning. Zhou Hongyi said, "The 360 model is ready to fight a lasting battle

What is the overall layout of 360 in the future? How does he view the argument that big models will replace traditional search as a search engine? How will security issues be addressed in the future? NetEase Technology had in-depth communication with Zhou Hongyi.

The scene is mature, one is open, and the other is open

Discussion on Layout: Two Things and Four Scenarios of 360

When it comes to the overall layout of 360, Zhou Hongyi summarized it with two points. One is to focus on the digital strategy, keep up with the times, and solidify the secure foundation of digitization. While addressing network security, it is also necessary to address data security and artificial intelligence security.

The second is to firmly grasp the core technology of the big model in one's own hands. Zhou Hongyi described the specific strategy as "two wings flying together". On the one hand, we will accelerate the self-development of core algorithms and build our own "engines". On the other hand, we will launch relevant products as soon as possible to seize user scenarios.

This is an industrial revolution level transformation. Whoever does not master the core technology of large models or the actual use of large models in practical scenarios will be eliminated by the industry, and basically all businesses will be reshaped. "Zhou Hongyi described the disruptive changes brought about by large model technology.

In the interview, he repeatedly emphasized the importance of scenes, stating that "a large model without scenes is lifeless." He proposed four scenario scenarios.

First, provide various tools for SMEs according to their fragmentation scenario needs. Secondly, collaborate with dark horses to jointly create industry GPTs. Thirdly, explore providing large models to governments and businesses. Fourthly, apply large models to intelligent hardware.

He pointed out that "our internal name is AllIn Big Model", and it is reported that 360's intelligent hardware department is exploring how to apply big models to intelligent hardware.

Big models cannot replace search

Talking about hallucinations: both advantages and disadvantages

While large models exhibit astonishing creativity, people also observe that they often have "illusions" that the generated content is not based on any real-world data, but rather on the big model's own imagination. This is what people usually call 'nonsense'.

Zhou Hongyi has publicly discussed and believed that this "nonsense" is a manifestation of the "intelligence" of the big model.

It is undeniable that ChatGPT is not perfect. For example, it often talks nonsense and provides answers that may be exaggerated. However, I believe these are all flaws that can be solved through increased training and user feedback. ChatGPT is not a search, the goal is to find accurate information without making mistakes. ChatGPT is not a simple output, it is based on learned knowledge and uses reasoning ability to organize problem answers Case. From this perspective, ChatGPT's' nonsense 'precisely reflects its intelligence. Having imagination and the ability to create stories out of thin air is an important symbol that distinguishes Homo sapiens from apes and other animals

But at the same time, he also believes that the fuzzy knowledge existing in large models is harmless if used for entertainment applications and written with open minds. But if it is used in work such as law, education, healthcare, or company affairs, knowledge ambiguity will become very serious.

For this reason, Zhou Hongyi, who has a background in search, denies the replacement of search by big models.

He emphasized: An important tool for solving knowledge ambiguity is search engines, so we are now turning search engines into a knowledge base. When it comes to knowledge accuracy issues, we can verify and correct their answers through search. Including search can also solve a timeliness problem at the same time, because large model training always has a training deadline, after which we cannot train again. So, the big language model cannot replace search, its search is actually relatively enhanced

There won't be just one big model

Talking about security: non development is the biggest insecurity

Why do you want to make your own big model? Zhou Hongyi spoke bluntly.

Firstly, not developing is the biggest insecurity. AI is an industrial revolution, and we cannot just give up on it because it has some safety issues. Secondly, if we don't do research and development on large models and don't understand, you just treat them as a black box and a thing you don't understand. How can you solve it? So, in the process of doing it, we not only have a research and development team working on it, but also our security capabilities understanding its principles Throughout the entire process, we may be able to propose better security solutions

He also does not deny the gap between China and abroad.

At present, there may be a real gap between China and GPT-4, and I think it is mainly in the so-called super strong emergence ability. This gap is not a gap in algorithms or models, but a gap in pre training data and training methods. We have some holes that we haven't fully explored

He predicted that in the future, China will not have only one big model. From an international trend, there will not be only one big model in the cloud. In the future, there will be different levels of big models. Some models will be deployed on computers, some models will be deployed on mobile phones, and some models may not be very large in scale, not necessarily hundreds of billions. The cost of hundreds of billions is very high, which can be tens of billions or tens of billions of dollars used in the industry

He believes that there is a high possibility of a large model with multiple small-scale large models in the future. Because a big model needs time and cost to train strong abilities in all aspects. However, in the professional field, what the big model needs is not irrelevant skills, but enough professional domain knowledge and understanding ability.

Finally, he emphasized the role of the scene. The big model still needs to find various scenarios. We said that combining with various scenarios, software, browsers, and search, Microsoft has had a relationship with the whole family bucket, and the whole family bucket has been transformed

And this is exactly what Zhou Hongyi is doing. (Yuan Ning)