<img src="https://github.com/Open-Source-O1/Open-O1/raw/main/picture/logo.jpg" width="200"/>

Open O1: A Model Matching Proprietary Power with Open-Source Innovation

🍓Homepage：OpenO1 🤗Huggingface: OpenO1 🤖Discord：OpenO1-Discord 🪩Slack：OpenO1-Slack

🌈Mission and Vision for future

Our Open O1 aims to match the powerful capabilities of the proprietary OpenAI O1 model, empowering the community with advanced open-source alternatives. Our model has been developed by curating a set SFT data for CoT Activation, which was then used to train both LLaMA and Qwen models. This training approach has endowed the smaller models with enhanced long-reasoning and problem-solving capabilities.

As the Open O1 project progresses, we will continue to push the boundaries of what's possible with large language models. Our vision is to create a model that not only achieves O1-like performance but also leads the way in test-time scaling, making advanced AI capabilities accessible to all. Through community-driven development and a commitment to ethical practices, Open O1 will be a cornerstone in the advancement of AI, ensuring that the future of technology is open and beneficial to all.

Demo

🤗Welcome to experience our model! Note: The model deployed on HuggingFace may have undergone quantization, which can lead to some performance degradation. If you'd like to experience the original performance of the model, you're encouraged to deploy it on your own infrastructure.🤗

🍓How many letters 'r' in the word 'strawberry'?

📋

Input

How many letters 'r' in the word 'strawberry'?

💡

Thought

Alright, I need to determine how many letters 'r' are in the word "strawberry." Let's start by looking at the word step by step. The word "strawberry" has several letters, and I'll list them out: s, t, r, a, w, b, e, r, r, y. Now, focusing on the letter 'r,' I can count each occurrence. 

First, there's an 'r' in the third position after the 's.' Then, there's another 'r' in the eighth position before the 'y.' Additionally, there's an 'r' in the ninth position, making it three 'r's in total. To be thorough, I'll double-check by counting again: 1st letter is 's,' 2nd is 't,' 3rd is 'r'—count, 4th is 'a,' 5th is 'w,' 6th is 'b,' 7th is 'e,' 8th is 'r,' 9th is 'r,' 10th is 'r,' and finally, 11th is 'y.' Yes, that confirms it—the letter 'r' appears three times.

But wait, could there be any other interpretations? Maybe considering uppercase versus lowercase, but since both 'r's in "strawberry" are lowercase, it doesn't change the count. Also, thinking about the pronunciation, even though 'r' might have a different sound, the number of 'r's remains the same visually. Another angle: if the word were spelled differently or had diacritics, but "strawberry" is straightforward in its spelling.

Is there a possibility of multiple meanings or contexts affecting the count? For example, in a different language or script, but assuming we're sticking to standard English spelling, the count should remain consistent. Also, considering the position of 'r's in the word, they are not repeated consecutively, which might affect certain linguistic analyses, but for the purpose of counting, their positions don't impact the total number.

In summary, after carefully examining each letter in "strawberry" and recounting to ensure accuracy, I'm confident that the letter 'r' appears three times in the word.

📝

Output

There are three letters 'r' in the word 'strawberry'.

🔔News

🚀 [11/21/2024] OpenO1 SFT data for CoT Activation release！Welcome to download on OpenO1-SFT data

🚀 [10/09/2024] OpenO1 model-v0.1 release！

🚀 [10/05/2024] OpenO1 initial release！

Content Navigation

Section	Description
💻 Model Deployments & Chat Templates	Instructions and examples for deploying models and using chat templates effectively.
✍ Example Demonstrations	Showcase of various use cases and practical demonstrations of the model's capabilities.
💯 System Performance	Analysis of system performance metrics, benchmarking results, and optimization strategies.
🎋 Training Details	An overview of the training process for Open O1, including datasets used, training methodologies, and any relevant hyperparameters.
❓ FAQ	Answers to frequently asked questions.
⚠️ Limitations	A discussion of the limitations of the models involved, including known issues, performance constraints, and areas for future improvement.

💻Model Deployment & 💬Chat templates

💻Model Deployment

To quickly get started with Open O1, we provide some steps for you to enjoy our model.

Model Download

• OpenO1-LLama-8B-v0.1
• OpenO1-Qwen-7B-v0.1

Deployment

1. Git clone this repository.

git clone https://github.com/OpenSource-O1/Open-O1.git

1. Install the dependencies listed in Deployment/requirements.txt.

2. Execute the script below.

python Deployment/app.py

1. You can choose OpenO1-Qwen-7B-v0.1 or OpenO1-LLama-8B-v0.1 model, which can be obtained in huggingface.

💬Chat templates

The chat templates of Open O1 is following LLaMA3.1's. The details can be obtained in chat-templates.

dialog = [
    {"role": "user", "content": "What's the weather like today?"},
    {"role": "assistant", "content": "It's sunny and warm, around 75°F."}
]

After using chat templates to process, the converted result is as follows：

<|begin_of_text|><|start_header_id|>user<|end_header_id|>
What's the weather like today?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
It's sunny and warm, around 75°F.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

💯System Performance

The following table provides a comprehensive comparison of the performance between llama3.1-8b-instruct and our model across multiple benchmarks. These evaluations were conducted in a zero-shot setting, meaning the models were tested without task-specific fine-tuning, highlighting their ability to generalize across diverse tasks. These benchmarks assess various aspects of reasoning, knowledge, and understanding in different domains, offering a clear indication of how each model handles complex tasks without prior exposure or specific task-related training. Our model consistently demonstrates competitive or superior performance, showcasing advancements in areas critical to reasoning, mathematical understanding, and general AI capabilities.

Model	GSM8K	MATH	MMLU	Hellaswag	ARC-C	BBH
llama3.1-8b-instruct	84.00	47.42	67.95	68.43	83.87	53.64
Ours(OpenO1-llama-8B-v0.1)	85.82	52.88	70.45	67.77	86.52	58.43

• GSM8K: Our model outperforms llama3.1-8b-instruct with a score of 85.82, demonstrating better reasoning ability in math word problems.
• MATH: It's important to note that the official score for llama3.1-8b-instruct on MATH is 51.9, but this was achieved in a CoT (Chain of Thought) setting. In our evaluation, we reproduced the result in a zero-shot setting, where llama3.1-8b-instruct scored lower at 47.42, while our model achieved 52.88, showing a significant improvement.
• MMLU: Our model leads with 70.45, indicating stronger general knowledge and understanding.
• Hellaswag: llama3.1-8b-instruct scores 68.43, slightly ahead of our model at 67.77.
• ARC-C: In ARC-C, our model reaches 86.52, outperforming llama3.1-8b-instruct.
• BBH: Our model achieves 58.43, surpassing llama3.1-8b-instruct’s score of 53.64.

The results highlights our model's superior performance in most benchmarks, with notable improvements in MATH, MMLU, ARC-C, and BBH.

🍭Available Models

• OpenO1-Qwen-7B-v0.1
• OpenO1-LLama-8B-v0.1

🎋Training Details

The training process for Open O1 utilizes the configuration settings from Llama Factory to optimize performance. This section includes details on the datasets used, training methodologies, and relevant hyperparameters.

method

stage: sft
do_train: true
finetuning_type: full
deepspeed: ds_z3_config.json

dataset

dataset: o1_style_dataset
template: llama3/qwen
cutoff_len: 4096
overwrite_cache: true
preprocessing_num_workers: 16

output

logging_steps: 10
save_steps: 1000
plot_loss: true
overwrite_output_dir: true

train

per_device_train_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 1.0e-5
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000

eval

val_size: 0.1
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 200

❓FAQ

Summarizing and updating...

⚠️Limitations

Open O1 is currently in its early stages of development. Open O1 primarily exhibits o1-like reasoning characteristics and broad search thinking capabilities. However, there is still significant progress to be made before it fully achieves O1 capabilities.

⭐Star History

Reference

• Learning to Reason with LLMs – A detailed blog post by OpenAI discussing methods to enhance reasoning abilities in large language models.

• OpenAI O1 Mini: Advancing Cost-Efficient Reasoning – An OpenAI blog that introduces the O1 Mini model and explores its efficiency in reasoning tasks.

• Awesome-LLM-Strawberry – A curated list of resources and tools related to large language models (LLMs) and reasoning capabilities, including O1.

✍Example Demonstrations

Reasoning Case

• Reasoning Case1
• Reasoning Case2

---

Coding

• Coding Case1
• Coding Case2

---

Math

• Math Case1
• Math Case2

---

Physics

• Physics Case1
• Physics Case2

---

Cipher

• Cipher Case1
• Cipher Case2

---

Counterfactual

• Counterfactual Case1
• Counterfactual Case2

---

Data Analysis

• Data Analysis Case1
• Data Analysis Case2

---

Administrative Aptitude Test(AAT)

• AAT Case1
• AAT Case2

---

Puzzle

• Puzzle Case1
• Puzzle Case2

---

Logic

• Logic Case1
• Logic Case2

---

Future Todo

Task	Estimated Schedule
Releasing our first version of SFT data that comprises o1-style thinking process	1~2 weeks
Reward model (and the corresponding data) for judging the thinking process of each model	2~3 weeks
Training infrastructure and pipeline for our o1-style data (both SFT and RLHF)	1 month
A new chatbot arena for evaluating and comparing the thinking process of different models	1 month
Reproducing the two o1 scaling laws both at training time (RLHF) and inference time	2~3 months

Disclaimer

We have not yet reached O1 level, but we are on the way. The first version has shown performance improvements on LLaMA and Qwen, with some promising data, but the style is still very close to O1. This project is non-profit. We welcome the entire community to join us in this journey. Moving forward, we will open source all the code and materials from the development process.

Citation

If you find our model, data, code useful, welcome to cite our paper

@article{
    supplement,
    title={},
    author={OpenO1 Team},
    journal={},
    url={},
    year={}
}

Acknowledgements(Updating)

This repo benefits from LLaMA-Factory, OpenRLHF, [awesome-o1](h