Chewy Lemon Cookie 11 B G G U F

Introduction

Chewy-Lemon-Cookie-11B-GGUF is a quantized text-generation model created by merging multiple pre-trained language models using Mergekit. It is optimized for various tasks, including roleplay and transformer-based text generation.

Architecture

The model is a combination of several pre-trained language models: SanjiWatsuki/Kunoichi-7B, SanjiWatsuki/Silicon-Maid-7B, KatyTheCutie/LemonadeRP-4.5.3, and Sao10K/Fimbulvetr-11B-v2. The merge process employs methods like passthrough and task arithmetic, utilizing configuration slices and varying weights for model parameters. The merged model is configured with bfloat16 data type.

Training

The model's performance has been evaluated on several datasets with tasks ranging from 0-shot to 5-shot scenarios. Metrics include strict accuracy, normalized accuracy, and exact match. Some notable results are:

• IFEval (0-Shot): 48.75% strict accuracy
• BBH (3-Shot): 33.01% normalized accuracy
• MATH Lvl 5 (4-Shot): 4.61% exact match

Guide: Running Locally

To run Chewy-Lemon-Cookie-11B-GGUF locally, follow these steps:

1. Clone the Repository: Download the model files from the Hugging Face model card.
2. Set Up Environment: Ensure you have Python and the necessary libraries installed, including transformers.
3. Load the Model: Use the Hugging Face transformers library to load the model.
4. Run Inference: Input text data and execute inference using the model.

For optimal performance, especially for large models like this, consider using cloud GPUs on platforms such as AWS, Google Cloud, or Azure.

License

The model is licensed under CC-BY-4.0, allowing for sharing and adaptation with appropriate credit.