Marco o1

Introduction

Marco-o1 is a large language model (LLM) designed to address both standard and open-ended questions across various disciplines, such as mathematics, physics, and coding. It emphasizes innovative reasoning strategies, aiming to generalize to broader domains with ambiguous standards and challenging reward quantification.

Architecture

Marco-o1 integrates techniques like Chain-of-Thought (CoT) fine-tuning, Monte Carlo Tree Search (MCTS), and novel reasoning action strategies. These methods are optimized for complex problem-solving tasks. MCTS utilizes confidence scores from model outputs to explore multiple reasoning paths, improving the model's problem-solving efficiency and accuracy. Additionally, the model has demonstrated enhanced reasoning capabilities in multilingual applications and translation tasks.

Training

The training of Marco-o1 involves full-parameter fine-tuning on the base model using an open-source CoT dataset, alongside self-developed synthetic data. The integration of MCTS further expands the solution space by guiding the search based on the model's output confidence. A reflection mechanism also prompts the model to self-reflect, enhancing its ability to solve complex problems.

Guide: Running Locally

1. Model Loading:

   from transformers import AutoTokenizer, AutoModelForCausalLM
   
   tokenizer = AutoTokenizer.from_pretrained("AIDC-AI/Marco-o1")
   model = AutoModelForCausalLM.from_pretrained("AIDC-AI/Marco-o1")
   

2. Inference:

   • Execute the inference script with customized inputs:

     ./src/talk_with_model.py
     

   • Alternatively, use the vLLM script:

     ./src/talk_with_model_vllm.py
     

For better performance, it is recommended to use cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

This project is licensed under the Apache License Version 2.0. The license allows for broad use, modification, and distribution under specified conditions. For detailed information, refer to the Apache License 2.0.