Qwen2.5 Math R M 72 B

Introduction

Qwen2.5-Math-RM-72B is engineered to enhance the Qwen2.5-Math model by providing detailed feedback on reasoning quality and intermediate steps, promoting robust improvements. It supports multilingual and multi-modal capabilities, offering preference signals in Chinese and English, and integrates various reasoning modes.

Architecture

Qwen2.5-Math-RM-72B offers guidance through a reward model that assesses the quality of responses. It utilizes a combination of data selection and reinforcement learning to improve model performance. The inference process is optimized through response sampling and Best-of-N strategies, which prioritize top-scoring responses for better results.

Training

The training process involves enhancing training data quality using reward model scoring and Rejection Sampling. Seamless integration into reinforcement learning provides effective reward signals, boosting model performance. The model also employs a "Best of N" inference strategy to yield improved results compared to majority voting methods.

Guide: Running Locally

To run Qwen2.5-Math-RM-72B locally, follow these steps:

1. Install Dependencies: Ensure you have transformers>=4.40.0 installed.
2. Load the Model: Use the Hugging Face transformers library to load the model:

   import torch
   from transformers import AutoModel, AutoTokenizer
   
   model_name = "Qwen/Qwen2.5-Math-RM-72B"
   device = "auto"
   
   model = AutoModel.from_pretrained(
       model_name, 
       device_map=device, 
       torch_dtype=torch.bfloat16,
       trust_remote_code=True,
   ).eval()
   
   tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
   

3. Inference Example: Use the model for inference as shown in the provided code snippet for a conversation.
4. GPU Recommendation: For optimal performance, running on cloud GPUs is recommended, such as those offered by AWS or Google Cloud.

License

This model is licensed under the Qwen license. For more information, refer to the license document.