calme 3.2 instruct 78b

Introduction

The CALME-3.2-INSTRUCT-78B model is an advanced iteration of the Qwen/Qwen2.5-72B, specifically fine-tuned for enhanced performance in generic domains. This model utilizes the ChatML prompt template and is designed to generate text based on given inputs.

Architecture

The model is based on the Qwen2.5-72B architecture, which was merged and fine-tuned on custom datasets to improve its functionality and expand its capabilities. Quantized versions of the model, such as the GGUF models, are also available.

Training

The model was evaluated using various datasets and metrics, achieving notable performance in different text generation tasks. For instance, it scored 80.63 on IFEval (0-Shot) and 62.61 on BBH (3-Shot). These results highlight the model's ability to handle diverse text generation challenges.

Guide: Running Locally

1. Setup Environment:

   • Install the Transformers library from Hugging Face.

   pip install transformers
   

2. Load the Model:

   • Use the pipeline method for simple integration.

   from transformers import pipeline
   
   messages = [{"role": "user", "content": "Who are you?"}]
   pipe = pipeline("text-generation", model="MaziyarPanahi/calme-3.2-instruct-78b")
   response = pipe(messages)
   

   • Alternatively, load the model and tokenizer directly.

   from transformers import AutoTokenizer, AutoModelForCausalLM
   
   tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/calme-3.2-instruct-78b")
   model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/calme-3.2-instruct-78b")
   

3. Hardware Recommendations:

   • For optimal performance, especially with large models, consider using cloud GPU services such as AWS, GCP, or Azure.

License

The CALME-3.2-INSTRUCT-78B model is distributed under the Qwen license. For more details, refer to the license document.