Cydonia 22 B v1.3 G G U F

Introduction

The Cydonia-22B-v1.3-GGUF model repository offers two GGUF quantizations of TheDrummer's Cydonia-22B-v1.3. These quantizations are designed to optimize model performance for specific computational environments.

Architecture

The model provides two quantizations:

• q6_K: A specific configuration for quantizing the model.
• q4_K_S: Another configuration with different quantization parameters. Both configurations utilize q8_0 for output and embedding tensors to ensure efficient computation and memory use.

Training

Details regarding the training process are not specified in the current documentation. However, the quantization approach suggests a focus on optimizing pre-trained models for efficient inference.

Guide: Running Locally

To run the Cydonia-22B-v1.3-GGUF model locally, follow these steps:

1. Clone the Repository: Use Git to clone the model repository from Hugging Face.
2. Install Dependencies: Ensure you have the required packages installed, potentially using pip.
3. Quantization Configuration: Choose between q6_K or q4_K_S based on your system's capabilities.
4. Run the Model: Execute the model using a suitable framework or script.

For optimal performance, especially with larger models, consider using cloud GPUs such as those available from AWS, Google Cloud, or Azure.

License

The model is distributed under a non-standard license labeled as "other." Users should refer to the repository's license section for specific terms and conditions.