Skywork o1 Open Llama 3.1 8 B

Introduction

The Skywork o1 Open model series, developed by the Skywork team at Kunlun Inc., introduces models with enhanced reasoning capabilities, such as slow thinking and planning. The series includes:

• Skywork o1 Open-Llama-3.1-8B: Enhanced reasoning chat model.
• Skywork o1 Open-PRM-Qwen-2.5-1.5B: Smaller-scale complex problem-solving model.
• Skywork o1 Open-PRM-Qwen-2.5-7B: Advanced reasoning capabilities for demanding tasks.

These models are designed to advance AI reasoning skills beyond previous state-of-the-art models.

Architecture

The Skywork o1 Open series utilizes a three-stage training process:

• Reflective Reasoning Training: Generates data for long-thinking tasks, followed by pre-training and fine-tuning.
• Reinforcement Learning for Reasoning: Uses the Skywork o1 Process Reward Model (PRM) to enhance reasoning.
• Reasoning Planning: Implements Tiangong's Q* online reasoning algorithm for optimal reasoning paths.

Training

The training focuses on enhancing cognitive abilities through proprietary methods, including multi-agent systems and reasoning reinforcement algorithms. The models show improvements in reasoning tasks across standard benchmarks.

Guide: Running Locally

To run the Skywork-o1-Open-Llama3.1-8B model locally:

1. Install Dependencies: Ensure you have Python and PyTorch installed.
2. Load Model and Tokenizer:

   import torch
   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "Skywork-o1-Open-Llama3.1-8B"
   model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   

3. Prepare Input: Format your input as a conversation for the model.
4. Generate Output:

   input_ids = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
   generation = model.generate(input_ids=input_ids, max_new_tokens=2048, do_sample=False, pad_token_id=128009, temperature=0)
   completion = tokenizer.decode(generation[0][len(input_ids[0]):], skip_special_tokens=True, clean_up_tokenization_spaces=True)
   print(completion)
   

5. Cloud GPUs: For efficient processing, consider using cloud GPUs such as AWS, GCP, or Azure.

License

Skywork models are distributed under the Skywork Community License, which supports commercial use. Users must adhere to the license terms, especially when using the models for commercial purposes. The models should not be used for activities that threaten security or engage in unlawful actions. The developers disclaim responsibility for issues arising from misuse or improper utilization of the models.