Skywork o1 Open P R M Qwen 2.5 7 B

Introduction

The Skywork o1 Open model series, developed by the Skywork team at Kunlun Inc., introduces models with enhanced reasoning capabilities, inspired by o1-like slow thinking. The series includes:

• Skywork o1 Open-Llama-3.1-8B: A chat model trained on Llama-3.1-8B, enhanced for reasoning.
• Skywork o1 Open-PRM-Qwen-2.5-1.5B: Focused on incremental process rewards for complex problem-solving.
• Skywork o1 Open-PRM-Qwen-2.5-7B: A more powerful model for demanding reasoning tasks.

These models exhibit innate thinking, planning, and reflecting abilities, showing significant improvements in reasoning skills on standard benchmarks.

Architecture

The Skywork-o1-Open-PRM series builds upon the Qwen2.5-Math-1.5B-Instruct and Qwen2.5-Math-7B-Instruct models. Evaluations include mathematical and code-related tasks across various datasets and settings.

Training

The models were trained and evaluated using standardized scripts and datasets from Qwen2.5-Math and Qwen2.5-Coder. Evaluations included datasets such as GSM8K, MATH, GaoKao, and others, using methods like Greedy Sampling and Best-of-N@64.

Guide: Running Locally

1. Clone the Repository:

   git clone https://github.com/SkyworkAI/skywork-o1-prm-inference.git
   cd skywork-o1-prm-inference
   

2. Install Dependencies:

   • For Hugging Face Inference:

     from transformers import AutoTokenizer
     

   • For VLLM Server:

     pip install vllm==v0.6.4.post1
     pip install -e .
     

3. Run Inference:

   • For Hugging Face:

     tokenizer = AutoTokenizer.from_pretrained(prm_model_path, trust_remote_code=True)
     # Prepare input and run the model
     

   • For VLLM Server:

     CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve /path/to/prm_model --host 0.0.0.0 --port 8081 --tensor-parallel-size 4 --gpu-memory-utilization 0.9 --enable-prefix-caching --dtype auto
     

4. Cloud GPU Suggestion: Consider using cloud platforms offering GPU instances, such as AWS EC2, GCP, or Azure, for efficient model inference.

License

Skywork models are available under the Skywork Community License, supporting commercial use. Users must comply with the license terms for any commercial applications. The models should not be used for activities threatening national security or unlawful actions.