AI Evolution Hits a "Data Wall"? Tech Giants Pivot to Applications: A Global Finance Weekly Review

AI Evolution Hits a "Data Wall"? Tech Giants Pivot to Applications: A Global Finance Weekly ReviewFor a long time, the artificial intelligence (AI) field has operated under the belief that "bigger data is better." However, recent industry reports suggest that large model evolution is encountering a "data wall

AI Evolution Hits a "Data Wall"? Tech Giants Pivot to Applications: A Global Finance Weekly Review

For a long time, the artificial intelligence (AI) field has operated under the belief that "bigger data is better." However, recent industry reports suggest that large model evolution is encountering a "data wall." OpenAI, Google, and Anthropic, among other giants, are experiencing setbacks in developing next-generation models, failing to achieve the breakthrough advancements of the past. This has sparked heated industry debate. OpenAI's former chief scientist, Ilya Sutskever, and other industry leaders have openly stated that the scaling law has reached its ceiling. While OpenAI CEO Sam Altman and others have refuted this claim, major companies are actively adjusting strategies, shifting their focus to application development, creating ripples across the global finance sector. This article delves into the AI evolution bottleneck, the strategic adjustments of tech giants, and a series of international financial headlines.

AI Evolution Hits a "Data Wall"? Three Major Models Stalled, Giants Shift to New Tracks

The development of artificial intelligence (AI) has always adhered to the belief that "more data equals better models." However, recent reports indicate that large model evolution has encountered a "data wall." OpenAI, Google, and Anthropic have all encountered bottlenecks in AI model development, with the improvements in new models significantly less pronounced than in previous generations. This news has sent shockwaves through the AI community. Turing Award winner Yann LeCun and former OpenAI chief scientist Ilya Sutskever, among other leading figures, point out that the scaling law for large models is nearing its limit, necessitating a change in thinking and training methods.

Meanwhile, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have countered the "bottleneck" theory. Nevertheless, companies like OpenAI and Google are actively adjusting their strategies, seeking solutions to insufficient training data and refining their application of the scaling law to maintain steady model development. Baidu CEO Robin Li believes that the slowdown in model development is not necessarily a bad thing. He suggests that rapid iteration in the early stages of foundational models is normal, but sustained for several years, it could destabilize the entire ecosystem. Meta CEO Mark Zuckerberg also stated that even with current technology, there's enormous room for growth at the application level.

OpenAI, Google, and Anthropic Encounter Development Bottlenecks

According to foreign media reports, the progress of OpenAI's next-generation flagship model (codenamed Orion) is far less significant than the improvement from GPT-3 to GPT-4, suggesting a slowdown in the pace of improvement. OpenAI is not alone in facing this bottleneck; its competitors, Google and Anthropic, have also reportedly encountered difficulties in developing advanced AI models. After rapidly releasing increasingly complex AI products, these three leading AI companies have found that the returns on their massive investments are diminishing. Sources say that Google's upcoming new version of Gemini has fallen short of internal expectations, and Anthropic has delayed the release of its Claude 3.5 Opus model.

The lack of high-quality artificial training data is a major challenge facing AI model development. Reportedly, Orion's performance on programming tasks is subpar, partly due to a lack of sufficient programming data for training. Even with improvements, it's difficult to justify the high cost of building and operating these models, which falls short of market expectations for significant upgrades.

Tech Leaders Debate the "Data Wall"

The news of the AI model development setbacks has sparked heated debate within the industry. Prominent AI commentator Gary Marcus has even declared "victory," arguing that GPT has entered a stage of diminishing returns. Marcus has long predicted a stagnation phase for generative AI. AI skeptics have consistently warned that simply scaling up large models is a limited approach. A year ago, Bill Gates expressed a similar sentiment, stating that current generative AI (AGI) is nearing its limit, and "GPT-5 won't be much better than GPT-4."

Former OpenAI chief scientist Ilya Sutskever also stated that the results of scaling pre-training have plateaued. Meta's chief scientist and Turing Award winner Yann LeCun retweeted Sutskever's statement, commenting, "I told you so." However, OpenAI CEO Sam Altman posted on social media that there is "no wall," suggesting that AI development has not encountered a bottleneck. Anthropic founder Dario Amodei also stated that the scaling law is based on empirical evidence rather than objective laws and believes it will continue to hold true.

Tech Companies Shift Strategies

Despite Altman's rebuttal of the "bottleneck" theory, OpenAI is actively exploring countermeasures. OpenAI has established a foundational team to explore how to address the scarcity of training data, adjust the application of the scaling law, and maintain the stability of model improvements. For example, OpenAI researchers have started using AI-generated synthetic data to train models, employing reinforcement learning methods, and incorporating post-pre-training reasoning, allowing the model more time to "think" when answering questions.

Google seems to be following a similar approach. DeepMind has assembled a team to develop capabilities similar to OpenAI's reasoning models and is focusing on manually improving models, including altering "hyperparameters."

Giants Pivot to AI Applications

Against the backdrop of the debate surrounding the AI evolution bottleneck, AI applications have experienced explosive growth. Thanks to its AI ad engine model Axon 2.0, ad platform company AppLovin significantly exceeded expectations, resulting in a sharp rise in its stock price. The AI platform significantly boosted Palantir's business growth, leading the company to announce a shift in its listing location from the New York Stock Exchange to Nasdaq. In the domestic AI application market, the number of users for ByteDance's Doubao and Kimi has seen a significant increase.

Baidu CEO Robin Li believes that a slowdown in AI model development is not a bad thing, suggesting that releasing major versions of foundational models every two years is a reasonable pace. Zuckerberg also stated that even without technological improvements, there is considerable room to build consumer and enterprise products based on existing technology. OpenAI plans to launch AI agent software next January to help users organize files and book tickets. Google is also developing similar AI agent software. Jensen Huang, CEO of NVIDIA, also expressed interest in AI applications, highlighting the potential of AI agents.

Nvidia's Earnings Report and the Future of AI Applications

Nvidia's earnings report is imminent, and Jensen Huang is expected to announce more about AI applications during the conference call. Dave Mazza, CEO of Roundhill Investments, stated that Nvidia needs to achieve new technological breakthroughs and sustained revenue growth in the AI field to maintain its leading market capitalization.

Trump Nominates Over 20 Key Positions, "Anti-Vaxxer" for Health Secretary? Pharma Stocks Plummet

US President-elect Trump announced numerous key appointments, including the nomination of Robert F. Kennedy Jr. as Secretary of Health and Human Services. Kennedy is a well-known "anti-vaccine advocate," a decision that has sparked concerns and questions from the American public, particularly among public health experts. Kennedy's nomination triggered market panic, causing several vaccine manufacturers' stock prices to fall for two consecutive days.

Gold Plunges, Bitcoin Soars

Hit by concerns over interest rate cuts, the US stock market plummeted, with the Nasdaq erasing most of its gains since the US election results. Gold prices fell sharply, marking the biggest weekly decline in three years. Bitcoin, however, has performed strongly, briefly exceeding $93,000, with institutions predicting a push towards $100,000 by the end of the year. Trump's support for cryptocurrencies is seen as a positive factor.

Putin and Scholz Hold First Phone Call in Nearly Two Years

Russian President Vladimir Putin and German Chancellor Olaf Scholz spoke by phone, their first conversation since December 2022. They discussed the conflict in Ukraine and the situation in the Middle East.

SpaceX to Sell Internal Shares, Valuation Soars to $250 Billion; xAI to Raise $6 Billion to Purchase 100,000 Nvidia Chips

SpaceX is preparing to sell existing shares, with a valuation exceeding $250 billion. Elon Musk's xAI is raising up to $6 billion in funding, primarily to purchase 100,000 Nvidia chips.

WHO Warns: Superbugs May Emerge

The World Health Organization warned that antimicrobial resistance is threatening humanity and could lead to the emergence of superbugs, increasing the risk of disease transmission, disability, and death.

Disclaimer: The content and data in this article are for reference only and do not constitute investment advice. Any actions taken based on this information are at your own risk.