Z E U S 8 B V10

Introduction

ZEUS-8B-V10 is a text generation model developed as a V2 recreation with minor updates, such as a unified tokenizer and preset seed for reproducibility. It integrates advanced techniques like int_mask and normalize via the mergekit tool.

Architecture

The model is a combination of multiple Llama-3.1 models, merged using the DARE TIES method. It leverages the base model unsloth/Meta-Llama-3.1-8B-Instruct with specific configurations to enhance performance.

Training

ZEUS-8B-V10 was trained using a merge method that integrates contributions from several models, including:

• akjindal53244/Llama-3.1-Storm-8B
• Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
• arcee-ai/Llama-3.1-SuperNova-Lite

The configuration involved parameters like int8_mask, normalize, and random_seed, with a layer range and model density/weight settings to optimize the merging process.

Guide: Running Locally

To run ZEUS-8B-V10 locally, follow these steps:

1. Set Up Environment: Install Python and the required libraries, primarily transformers and mergekit.
2. Download Model: Retrieve the model from the Hugging Face model hub.
3. Load and Configure: Use the transformers library to load the model and tokenizer, setting parameters according to your needs.
4. Inference: Perform text generation tasks using the model.

For optimal performance, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

The model and its components are subject to specific licenses, which should be reviewed on the Hugging Face model card or repository to ensure compliance with usage terms.