West Kunai Hermes 10.7b test G G U F

Introduction

The WestKunai-Hermes-10.7B model is a test version using GGUF quantization, primarily in English. Developed by mradermacher, it utilizes Transformers and MergeKit libraries and is licensed under CC BY-NC 4.0.

Architecture

This model is based on the GGUF architecture, designed for efficient inference and deployment. It is a quantized version of the base model, seyf1elislam/WestKunai-Hermes-10.7b-test, and offers various quantized forms sorted by size and performance.

Training

The model is derived from the seyf1elislam/WestKunai-Hermes-10.7b-test base, focusing on static quantization methods. Weighted/imatrix quants are not currently available but can be requested through community discussions.

Guide: Running Locally

1. Preparation: Ensure you have the Transformers library installed.
2. Download: Obtain the desired quantized GGUF file from the provided links.
3. Setup: Follow TheBloke's README for guidance on using GGUF files, especially for concatenating multi-part files.
4. Execution: Use a Python environment to load and run the model.

For enhanced performance, consider using cloud GPUs like those from AWS or Google Cloud.

License

The model is released under the Creative Commons BY-NC 4.0 license, allowing for non-commercial use with proper attribution.