OpenAI Launches New Model o1: Revolutionary Advancements with Deep "Reasoning" Capabilities

OpenAI Launches New Model o1: Revolutionary Advancements with Deep "Reasoning" CapabilitiesOpenAI recently unveiled two variants of its new model, OpenAI o1 o1-preview and o1-mini which have garnered significant attention due to their breakthroughs in "reasoning" capabilities. Artificial intelligence expert Simon Willison delves into the design principles and implementation details of these novel models, unveiling their unique technical advantages and shortcomings

OpenAI Launches New Model o1: Revolutionary Advancements with Deep "Reasoning" Capabilities

OpenAI recently unveiled two variants of its new model, OpenAI o1 o1-preview and o1-mini which have garnered significant attention due to their breakthroughs in "reasoning" capabilities. Artificial intelligence expert Simon Willison delves into the design principles and implementation details of these novel models, unveiling their unique technical advantages and shortcomings.

o1 is not merely an upgrade of GPT-4o but rather a rebalancing of costs and performance by enhancing the model's "reasoning" prowess. This breakthrough stems from OpenAI's in-depth research on "chain of thought," elevating the previous strategy of "step-by-step thinking" to new heights.

1. Chain of Thought Training: The Key to Enhanced Reasoning

OpenAI declares, "We've developed a new set of AI models that are designed to spend more time thinking before they respond." These new models can be viewed as a deeper extension of chain-of-thought prompting strategies, embodying the "step-by-step thinking" approach.

OpenAI's paper, "Learning to Reason with LLMs," elaborates on the training methodology behind o1, revealing the secrets behind its enhanced capabilities. The paper highlights OpenAI's utilization of large-scale reinforcement learning algorithms and meticulously designed training processes to enable the model to leverage data effectively and proficiently employ chain of thought for in-depth contemplation.

Through reinforcement learning training, o1 not only learns to optimize chain-of-thought usage but also acquires critical self-improvement skills. o1 can identify and rectify errors, breaking down complex problems into a series of more manageable sub-tasks. When existing methods fail, it explores alternative approaches until the optimal solution is found. This process significantly bolsters the model's reasoning abilities.

In essence, o1 models have achieved a qualitative leap in handling complex prompts. Faced with tasks requiring retrospection and in-depth "thinking," the model showcases superior performance, transcending mere reliance on next-token prediction.

2. API Documentation Unveils Underlying Details and Usage Limitations

OpenAI provides a series of intriguing details about the new models and their design trade-offs through API documentation.

• Application Scenario Selection: For applications reliant on image inputs, function calls, or prioritizing immediate response speeds, GPT-4o and its streamlined version, GPT-4omini, remain ideal choices. If projects necessitate deep reasoning capabilities and can accommodate longer response times, then o1 models are undoubtedly the better option.

• API Access Permissions: Currently, access to o1-preview and o1-mini is restricted to Tier 5 account users, and unlocking access requires accumulating at least $1,000 in API credits.

• System Prompt Limitations: The model integrates with the existing chat completion API but only supports message interactions between users and assistants, lacking support for system prompts.

• Other Feature Limitations: The current model does not offer streaming support, tool integration, batch processing calls, or image inputs.

• Response Time: Considering the varying amounts of reasoning required for the model to solve problems, the time taken to process requests can range from a few seconds to several minutes.

• Introduction of Reasoning Tokens: These tokens remain invisible in API responses but play a crucial role, serving as the driving force behind the new model's capabilities and are counted and billed as output tokens. Given the importance of reasoning tokens, OpenAI recommends budgeting approximately 25,000 reasoning tokens for prompts to fully leverage the new model.

• Increased Output Token Quota: o1-preview's quota has been raised to 32,768 tokens, while o1-mini boasts a quota of 65,536 tokens. Compared to GPT-4o and its mini version (both with a quota of 16,384 tokens), this increase provides users with more resources.

• RAG Prompt Optimization: When integrating additional context or documentation, rigorous filtering should be applied, retaining only the most relevant information to prevent the model from generating overly complex responses. This diverges from the traditional RAG approach, which tends to incorporate a vast array of potentially relevant documents into prompts.

3. Hidden Reasoning Tokens Spark Controversy

Regrettably, reasoning tokens remain hidden within API calls. Users are charged for these tokens without insight into their specific content. OpenAI justifies this policy, stating, "The intent behind hiding the chain of thought is to ensure the model's 'thinking' process remains independent and free from external interference or manipulation of its reasoning logic. Displaying the complete chain of thought might expose inconsistencies and impact user experience."

This decision stems from multifaceted considerations: ensuring security and policy adherence, as well as maintaining a technological competitive edge to prevent rivals from utilizing reasoning outcomes for training purposes.

As a party holding reservations about large language model development, Willison expresses dissatisfaction with this decision. He argues that maintaining explainability and transparency is paramount alongside pursuing technological innovation. Concealing key details signifies a weakening of transparency, feeling like a retrograde move.

4. Example Interpretation: Showcasing o1's Reasoning Abilities

OpenAI provides numerous examples within the "chain of thought" section, including Bash script generation, crossword puzzle solving, and chemical solution pH value calculation, offering initial glimpses of these models' chain-of-thought capabilities within the ChatGPT user interface. It does not, however, display the raw reasoning tokens but rather simplifies complex reasoning steps into comprehensible summaries through an optimization mechanism.

OpenAI also furnishes two additional documents showcasing more complex examples. In "Using Reasoning for Data Validation," o1-preview demonstrates generating examples within an 11-column CSV dataset and verifying data accuracy through various strategies. Conversely, "Using Reasoning to Generate Procedures" illustrates how to transform knowledge base articles into standardized operational processes that large language models can parse and execute.

Willison has also solicited numerous prompt cases on social media where GPT-4o failed to deliver but o1-preview excelled. Two instances stand out:

• Word Count Challenge: "How many words are in your answer to this prompt?" The o1-preview model takes approximately ten seconds, undergoing five reasoning processes before responding, "There are seven words in this sentence."

• Humor Interpretation: "Explain this joke: 'Two cows are standing in a field. One cow asks the other, 'What do you think about mad cow disease?' The other one says, 'I don't care, I'm a helicopter.'" o1-preview provides both plausible and detailed explanations, while other models remain stumped.

Nevertheless, high-quality examples remain scarce. OpenAI researcher Jason Wei points out that although o1 demonstrates impressive performance on AIME and GPQA tests, this effectiveness isn't always readily apparent. Finding prompts that stump GPT-4o but elicit outstanding performance from o1 is challenging, but when discovered, o1's capabilities are truly magical. We all need to seek more challenging prompts.

On the other hand, Ethan Mollick, a Wharton School management professor and artificial intelligence expert, offers an initial evaluation of o1 based on several weeks of preview experience. He highlights a crossword puzzle example where the o1-preview model displays clear reasoning steps, noting inconsistencies between 1Across and 1Down's initial letters, proactively suggesting answer replacements to ensure coherence.

5. The Future of Reasoning Models: Opportunities and Challenges Await

This new advancement in the field of artificial intelligence presents numerous puzzles to be solved and potential opportunities awaiting exploration by the community to identify the optimal application scenarios for these models.

During this period, Willison anticipates that GPT-4o (along with models like Claude 3.5 Sonnet) will continue to play a significant role. Simultaneously, we will witness how these reasoning models expand our thought patterns, addressing previously insurmountable tasks.

Furthermore, Willison anticipates other AI labs, particularly those within the open model weights community, to actively follow suit, leveraging their unique model versions to replicate and refine these chain-of-thought reasoning outcomes.