Captain Eris Diogenes_ Twilight V0.420 12 B G G U F

Introduction

The Captain-Eris-Diogenes_Twilight-V0.420-12B-GGUF model is a quantized version of the original model developed by Nitral-AI. It is created using llama.cpp and is available on the Hugging Face platform. This model combines features from two base models: Nitral-AI/Captain-Eris_Twilight-V0.420-12B and Nitral-AI/Diogenes-12B-ChatMLified.

Architecture

The model architecture includes two main components:

• Nitral-AI/Captain-Eris_Twilight-V0.420-12B
• Nitral-AI/Diogenes-12B-ChatMLified

These components are merged using a spherical linear interpolation (slerp) method. The YAML configuration specifies that both models contribute equally across a layer range of [0, 40].

Training

The training involves the following parameters:

• slices: Model layer ranges for merging.
• merge_method: slerp.
• parameters: Include specific filter values for self_attn and mlp, and a constant t value set to 0.420.
• dtype: bfloat16 for reduced precision and memory usage.

Guide: Running Locally

To run this model locally, follow these steps:

1. Clone the Repository: Download the model files from Hugging Face.
2. Install Dependencies: Use pip to install the required libraries, including transformers.
3. Load the Model: Utilize the Hugging Face transformers library to load the model.
4. Run Inference: Use the model to generate predictions or perform the specific task it was trained for.

Suggested Cloud GPUs

For optimal performance, consider using cloud-based GPUs such as NVIDIA A100 or V100 available on platforms like AWS, Google Cloud, or Azure.

License

The model is provided under a license that permits its use and modification. Users should refer to the Hugging Face model card for specific licensing details and adhere to any restrictions or obligations outlined.