magnum v4 72b

Introduction

Magnum V4-72B is a model series designed to emulate the prose quality of Claude 3 models, such as Sonnet and Opus. It is experimentally fine-tuned on top of Qwen2.5-72B-Instruct, yielding impressive results.

Architecture

The model is based on the Qwen2.5-72B-Instruct architecture, utilizing Axolotl's configuration and leveraging LigerPlugin optimizations. It supports text generation tasks, particularly in conversational and chat scenarios. The model is compatible with the Transformers library and utilizes datasets from Anthracite and other contributors.

Training

Training was conducted using 8x mi300x GPUs, sponsored by DoctorShotgun. The training process involved full-parameter fine-tuning, leveraging datasets such as anthracite-org/c2_logs_32k_llama3_qwen2_v1.2 and lodrick-the-lafted/kalo-opus-instruct-3k-filtered. The process utilized Axolotl's framework and various plugins to enhance model performance.

Guide: Running Locally

1. Install Dependencies: Ensure you have Python and the necessary libraries installed, such as Transformers and Safetensors.
2. Download the Model: Obtain the model files from Hugging Face's model hub.
3. Load the Model: Use the Transformers library to load the model and tokenizer.
4. Run Inference: Input text using the provided prompt formats to generate responses.

Suggested Cloud GPUs: To efficiently run the Magnum V4-72B model, consider using cloud services offering powerful GPUs, such as AWS with NVIDIA A100 or Google Cloud's TPU.

License

The Magnum V4-72B model is licensed under the Apache-2.0 License, allowing for broad use and modification with proper attribution.