OpenAI's ChatGPT Search: Promising Future, Not Yet a Google Killer

OpenAI's ChatGPT Search: Promising Future, Not Yet a Google KillerOpenAI last week officially launched its highly anticipated search product, ChatGPT Search, posing a direct challenge to Google. This moment was anticipated by the industry, prompting Google to integrate AI-generated answers into its core products earlier this year

OpenAI's ChatGPT Search: Promising Future, Not Yet a Google Killer

OpenAI last week officially launched its highly anticipated search product, ChatGPT Search, posing a direct challenge to Google. This moment was anticipated by the industry, prompting Google to integrate AI-generated answers into its core products earlier this year. However, OpenAI's search engine has been touted by some as a potential "Google killer," due to several embarrassing "hallucinations" that emerged.

However, after using ChatGPT Search as his default search engine for about a day, tech journalist Maxwell Zeff quickly switched back to Google. While, in some ways, OpenAI's search product is impressive, offering a glimpse into the future of AI-powered search, it's still too idealistic for daily use.

Zeff frequently felt frustrated during his ChatGPT Search testing, realizing how many keyword searches he conducts daily. He couldn't reliably find what he needed on ChatGPT Search using short queries. For the first time in years, Zeff genuinely craved returning to Google Search. Of course, this isn't to say that Google's search quality hasn't declined over the past decade; in fact, it has due to the influence of ads and search engine optimization. But during his testing, Zeff had to keep Google open in another window because ChatGPT Search often failed to provide him with the correct answers or page links.

Who wins: ChatGPT Search or short queries? Zeff tried to check on the NBA game between the Denver Nuggets and Minnesota Timberwolves by entering "Nuggets score." However, ChatGPT responded that the Nuggets were winning, but they were actually losing. Worse, according to Google results, the score for the Timberwolves shown by ChatGPT was 10 points lower than the actual score. In another instance, Zeff typed in "earnings today" to find out about company quarterly earnings reports that might impact Friday's stock prices. ChatGPT told him that Apple and Amazon would report their earnings on Friday, when in fact, both companies had already released their reports a day earlier. In essence, ChatGPT "hallucinated" and provided incorrect information.

During another test, Zeff entered the name of a tech executive, hoping to find his contact information. ChatGPT not only displayed a summary of the executive's Facebook profile but also "fabricated" a link to his LinkedIn page. However, when Zeff clicked the link, he received an error message. On another occasion, Zeff entered "loose jeans" hoping to find options for buying them. ChatGPT Search first explained what loose jeans were (though this definition might be redundant for most) and then suggested he buy a nice pair from Amazon. There are numerous other examples, but I think you get the picture. On his first day using ChatGPT Search, Zeff encountered many dead links, informational "hallucinations," and random answers.

Maybe a "Google killer" in the future, but not today. For OpenAI, the release of ChatGPT Search is a landmark event. While often downplaying his startup's AI capabilities, OpenAI CEO Sam Altman has been effusive about ChatGPT Search, calling it "really good." This time, the reason for the excitement might be the significance of search as one of the biggest businesses on the internet, and OpenAI's search capabilities genuinely have the potential to pose a substantial threat to its biggest competitor, Google.

To be objective, Google Search is a 25-year-old product, while ChatGPT Search is entirely new. OpenAI states in a blog post that they plan to continuously improve the feature based on user feedback in the coming months, making it a crucial area of investment for the startup.

To be sure, ChatGPT Search performs admirably in answering complex and lengthy research questions. For example, for a query like "Which US professional sports league is the most diverse?" Google might struggle to provide a satisfactory answer quickly, but ChatGPT Search can swiftly synthesize information from multiple websites and provide a fairly comprehensive answer within seconds. Similarly, Perplexity Search, launched over a year ago, excels in this category. Compared to traditional ChatGPT, ChatGPT Search now has web access, offering users a better web browsing interface. Now, ChatGPT provides clearer links to sources of informationfor news reports, ChatGPT will collaborate with major news media companies that have already entered into all licensing agreements.

However, most Google Search queries are not so lengthy. To truly replace Google, OpenAI needs to improve ChatGPT Search's performance with more practical and shorter daily searches. OpenAI doesn't shy away from acknowledging this shortcoming in ChatGPT Search. Niko Felix, a company spokesperson, stated, "With ChatGPT Search, we've seen users are more likely to ask questions in a more natural, conversational way, in contrast to how they used other search tools in the past. At the same time, we've also noticed that those short, web navigation queriesoften only a few wordsare also very common. We plan to continue to optimize in the future to improve the user experience for such queries."

In other words, these short keyword queries are Google's enduring advantage, and until OpenAI can solve them flawlessly, Google will remain the preferred search tool for many. OpenAI might face more challenges when handling these short queries for several reasons:

Firstly, ChatGPT technically relies on Microsoft Bing, which is generally considered to be inferior in search quality to Google. Secondly, large language models might not be very good at handling these short inputs. These models typically require complete written prompts to generate effective answers. Perhaps we need to find a way to re-guide usersuntil ChatGPT Search can excel at these searches, using large language models to transform short queries into more detailed questions.

It's worth noting that while OpenAI's search product is newly launched, another AI search tool, Perplexity, already handles 100 million search queries weekly. Perplexity has also been touted as a "Google killer" but faces similar issues with short queries. Aravind Srinivas, CEO of Perplexity, has discussed how people use his product differently compared to Google Search. He said, "The average query length in Google Search is typically between 2 and 3 words, whereas in Perplexity, the average query length is between 10 and 11 words. This suggests that people on Perplexity are more likely to ask direct questions. In contrast, on Google, users only need to enter a few keywords to quickly jump to relevant links."

Zeff believes that the fact that people don't use these new products for web navigation reveals a more significant problem than OpenAI or Perplexity publicly acknowledge. It means that ChatGPT Search and Perplexity haven't been able to replace Google Search's core advantageweb navigation. Instead, these AI products are carving out a new niche, uncovering and presenting information that has been overlooked or hidden in traditional searches. Of course, this is a highly valuable niche in itself.

Both OpenAI and Perplexity claim they are confident in improving their handling of these short queries in the future. But until then, Zeff believes both products are far from replacing Google entirely. If OpenAI truly wants to become the new gateway to the internet, it must create a better, more convenient entrance than the existing one.