The rise of open-source artificial intelligence models, catching up with ChatGPT and Google, what is its advantage

On May 18th, it was reported that Google and OpenAI are developing proprietary artificial intelligence models, but free open source models are also proliferating. Google employees say that the company is facing increasing pressure and if it does not release more open source models, it may fall behind in this artificial intelligence competition

On May 18th, it was reported that Google and OpenAI are developing proprietary artificial intelligence models, but free open source models are also proliferating. Google employees say that the company is facing increasing pressure and if it does not release more open source models, it may fall behind in this artificial intelligence competition. In addition, there are rumors that OpenAI is preparing to release a new open source language model. So, what are the advantages of open source models?

The following is the translated text:

In February of this year, Meta provided a batch of advanced machine learning models to the academic community, which can understand natural language conversations. This measure has sparked a wave of artificial intelligence development. In just a few weeks, scholars have transformed these models into open source software and launched free products that can replace ChatGPT and other proprietary artificial intelligence software (which has limitations on use and modification).

Ian Schmidt, a computer science professor at the University of California, Berkeley, stated that the free artificial intelligence model is currently "quite close" in performance to proprietary models from Google and OpenAI, and most software developers will ultimately choose to use the free model. Stoica also used Meta's technology to develop a key open source artificial intelligence model.

If Stoica's viewpoint is correct, then open source artificial intelligence will overturn the business plans of companies such as Google, OpenAI, and Microsoft. Anyone can access powerful artificial intelligence tools at a very low cost, and Meta can also benefit from them.

Stoica et al. utilized Meta's research findings to develop Vicuna, an open source language understanding model that was released in March of this year. Vicuna used ChatGPT data, which comes from conversations between users and OpenAI chat robots on a certain website. The rapid progress of open-source artificial intelligence, as well as Vicuna's outstanding performance, has sparked warnings from Google's senior engineer Luke Sernau. He warned colleagues that if the company continues to focus on proprietary software to catch up with OpenAI, the company will face the risk of falling behind.

In an internal memo, Seno wrote, "If there were a high-quality, unrestricted free alternative, who would be willing to buy a product that we have usage restrictions on?" He also said that the development of open source artificial intelligence has "surpassed us," so "Google should become the leader of the open source community," and "give up some control over our model. (He did not respond to the request for comment in this article.)

This memo resonates with the entire industry, including some Google employees. Although Seno may have exaggerated the capabilities of open source artificial intelligence, underestimated its costs and other risks, most AI practitioners agree with the memorandum's conclusion that Meta will benefit from publishing its model. Meta uses artificial intelligence models internally for content recommendation and advertising positioning. As developers improve the models released by Meta, Meta will also be able to incorporate these improvements into its internal artificial intelligence.

At an analyst conference call in April, when asked about the company's artificial intelligence strategy, Meta CEO Mark Zuckerberg said, "If the industry standardizes the basic tools we are using, it would be even better. We can benefit from the improvements of others

UC Berkeley graduate student Zheng Pity participated in the development of Vicuna, stating that the models released by Meta are collectively referred to as LLaMA and cannot be directly used for commercial purposes. But Meta provided researchers with sufficient information to create similar models for commercial applications.

Google is not completely exclusive to its AI software. As early as 2020, before the emergence of ChatGPT, Google released the open source language model T5, allowing developers to build software that can perform tasks such as translating and writing abstracts. Subsequently, Google released a more advanced version of Flan-T5. But according to Stoica and other practitioners, the software released by Meta has brought significant improvements, surpassing the level that Google models can achieve, which makes engineers more inclined to use models based on Meta software.

The Open Source Model of OpenAI

According to insiders, OpenAI is preparing to publicly release a new open source language model, which may increase pressure for Google, which wants to play a greater role in the field of open source artificial intelligence. It is currently unclear whether OpenAI plans to regain the momentum of Vicuna or other software developed based on the Meta model through open source software. But it is unlikely to release models that compete with GPT. The valuation of OpenAI reached $27 billion, mainly due to its proprietary model with more commercial value rather than open source model, although the first two versions of GPT were open source. A spokesperson for OpenAI did not respond to a request for comment.

Open source models like Vicuna may only cost a few hundred dollars to train, and users can choose to avoid paying high fees to software developers.. In contrast, Google, OpenAI, and Microsoft have been selling their proprietary models, which businesses can use to automate customer service, outline medical research, or generate marketing copy for various purposes. Last year, Microsoft began selling OpenAI models that it had invested billions of dollars in, while Google and Amazon began selling their models to external developers this year.

However, in recent weeks, an increasing number of open source alternatives have emerged. In addition to Vicuna based on Meta software, engineers can also choose other models from German non-profit organizations LAION and startups including Databricks and StabilityAI. Stoica created a website to try to measure the quality of these open source models compared to proprietary models such as OpenAI's GPT-4. (Stoica co founded Databricks, which sells software that enables data scientists to use artificial intelligence.)

According to Stoica, Google still has two major advantages compared to open source software. He said that if Google utilizes its user database, its model may perform better for certain specialized purposes, such as content recommendations, which are inaccessible to outsiders. (However, a Google spokesperson clarified that the company will not use existing user data to train its basic artificial intelligence models.)

In addition, Stoica pointed out that Google has extensive expertise in managing large-scale computer infrastructure, which means it can run artificial intelligence software models at lower costs, including providing services to its cloud clients. Last week, Google announced a series of improvements to the Bard model, which is Google's solution to counter ChatGPT.

Meanwhile, OpenAI collects data from a large number of users' interactions with ChatGPT, taking the lead in improving artificial intelligence software. In addition, this company has also reached a private agreement with Microsoft to use Microsoft's computing infrastructure.

Open source artificial intelligence software can allow more companies to use proprietary data to solve their problems. Stoica provides an example where an airline can use the records of millions of customer service calls to create automatic responses. Bloomberg stated in March that they used their own data to train machine learning models to better understand financial information. In addition, according to Vicuna developers on the Discord server, engineers have been trying to use it for creative writing and programming.

Stoica stated that he and his colleagues are working to increase the number of calculations in the Vicuna model to improve its ability in inference tasks, such as writing code. Vicuna's development team is a branch of Sky Computing Laboratory located in Berkeley, which receives millions of dollars in annual budgets from listed companies such as Microsoft, Google, and Amazon, each providing approximately $500000.

Open source artificial intelligence software has thwarted OpenAI's ambitions. Last April, OpenAI released Dall-E2, an artificial intelligence program that generates raw images using text descriptions. However, the rapid rise of an open source alternative called StableDiffusion has caused waves in the field of artificial intelligence, even to the surprise of OpenAI employees. In the end, Dall-E-2 did not become the dominant player in this field.

I have reason to believe that the big language model will also follow the same pattern, "Stoica said. (Small)