Z E U S 8 B V13

Introduction

ZEUS-8B-V13 is a text generation model that leverages multiple merging techniques and models to optimize performance across various datasets. It is part of the Hugging Face model repository and is compatible with the Transformers library.

Architecture

The model was created using the DARE TIES merge method, which combines several models: unsloth/Meta-Llama-3.1-8B-Instruct, arcee-ai/Llama-3.1-SuperNova-Lite, unsloth/Llama-3.1-Storm-8B, and Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2. The configuration involves a base model with bfloat16 data type and specific parameters for int8 masking, normalization, and random seed. The model's architecture is divided into slices, each drawing from different models with designated layer ranges and density weights.

Training

ZEUS-8B-V13 was evaluated on multiple datasets, including IFEval (0-Shot), BBH (3-Shot), MATH Lvl 5 (4-Shot), GPQA (0-shot), MuSR (0-shot), and MMLU-PRO (5-shot). The model demonstrated varying levels of accuracy, with the highest being 79.04% on the IFEval dataset.

Guide: Running Locally

To run ZEUS-8B-V13 locally, follow these steps:

1. Install Python and Pip: Ensure you have Python and Pip installed on your machine.
2. Install Transformers Library: Use the command pip install transformers to get the necessary library.
3. Download the Model: You can download the model from the Hugging Face model repository.
4. Load the Model: Use the Transformers library to load the model into your Python environment.
5. Run Inference: Use the model to perform text generation tasks.

For optimal performance, consider using cloud GPUs such as those provided by AWS, Google Cloud, or Azure.

License

ZEUS-8B-V13 is licensed under llama3.1, which dictates the terms and conditions for using the model. Users should review this license to ensure compliance with its provisions.