O L Mo 2 1124 13 B R M

Introduction

OLMo-2-1124-13B-RM is a variant of the OLMo-2 13B model developed by the Allen Institute for AI. It is designed for advanced text generation tasks, leveraging supervised fine-tuning and DPO training on a customized dataset. The model is primarily configured for English language applications and utilizes the Apache 2.0 license.

Architecture

OLMo-2-1124-13B-RM is part of a series of open language models, trained on diverse datasets including publicly available, synthetic, and human-created data. The model architecture is based on the transformers library and is optimized for text generation tasks. It is a post-trained variant with specific adjustments in its training process.

Training

The model underwent supervised fine-tuning on an OLMo-specific variant of the Tülu 3 dataset and further training to enhance its performance on various tasks such as MATH, GSM8K, and IFEval. The training process aims to deliver state-of-the-art results across a range of applications, including conversational interfaces.

Guide: Running Locally

1. Install Dependencies: Ensure you have Python and Git installed. Use pip to install the transformer library from Hugging Face's main branch:

   pip install --upgrade git+https://github.com/huggingface/transformers.git
   

2. Load the Model: Use the following Python snippet to load the model:

   from transformers import AutoModelForSequenceClassification
   olmo_reward_model = AutoModelForSequenceClassification.from_pretrained("allenai/OLMo-2-1124-13B-RM")
   

3. Resources: For optimal performance, consider using a cloud GPU from providers such as AWS, Google Cloud Platform, or Azure.

License

OLMo-2-1124-13B-RM is distributed under the Apache 2.0 license, which permits use for research and educational purposes. Additional terms may apply due to the use of third-party model outputs. For detailed guidelines, refer to the Responsible Use Guidelines by the Allen Institute for AI.