magnum v4 27b

Introduction

The Magnum-V4-27B model by Anthracite-ORG is designed for high-quality text generation, fine-tuned to replicate the prose quality of Claude 3 models, such as Sonnet and Opus. It is built on top of Gemma 27b, optimized for chat applications.

Architecture

• Base Model: IntervitensInc/gemma-2-27b-chatml
• Model Type: AutoModelForCausalLM
• Tokenizer Type: AutoTokenizer
• Library: Transformers
• Language: English
• Tags: Chat, Conversational
• Pipeline Tag: Text Generation

Training

The model underwent full-parameter fine-tuning over 2 epochs using 8 NVIDIA H100 GPUs. The training utilized various datasets, including ShareGPT, with a sequence length of 8192 tokens. Training was facilitated by Recursal AI and Featherless AI, with the process managed by the Axolotl framework.

Guide: Running Locally

1. Setup Environment:

   • Ensure you have Python and required libraries installed, such as transformers and torch.

2. Download Model:

   • Use the Hugging Face Hub to download the model: anthracite-org/magnum-v4-27b.

3. Load Model:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model = AutoModelForCausalLM.from_pretrained("anthracite-org/magnum-v4-27b")
   tokenizer = AutoTokenizer.from_pretrained("anthracite-org/magnum-v4-27b")
   

4. Inference:

   • Prepare input prompts and use the model for text generation:

   inputs = tokenizer("Your prompt here", return_tensors="pt")
   outputs = model.generate(inputs["input_ids"])
   print(tokenizer.decode(outputs[0]))
   

5. Cloud GPUs:

   • For optimal performance, consider using cloud GPU services like AWS, GCP, or Azure.

License

The model is released under the Gemma license. It is important to review the license terms to understand usage rights and restrictions.