miscii 14b 1225 G G U F

Introduction

MISCII-14B-1225-GGUF is a text generation model that supports English and Chinese languages. It has been quantized using the llama.cpp library for efficient performance and is part of the custom research category. The model is licensed under Apache 2.0.

Architecture

The model is based on the sthenno-com/miscii-14b-1225 architecture and has undergone various quantization methods to optimize performance on different hardware configurations. The quantized versions are designed to be used with LM Studio and are tailored to run efficiently on both CPUs and GPUs.

Training

The model has been evaluated on several datasets using few-shot learning techniques. It achieved a strict accuracy of 78.78 on IFEval with 0-shot learning, normalized accuracy of 50.91 on BBH with 3-shot learning, and an exact match score of 31.57 on MATH Lvl 5 with 4-shot learning. The results demonstrate its capability in handling different levels of complexity in text generation tasks.

Guide: Running Locally

1. Install Hugging Face CLI:

   pip install -U "huggingface_hub[cli]"
   

2. Download the Model: Choose a quantization file that suits your system’s RAM or VRAM capacity. For instance:

   huggingface-cli download bartowski/miscii-14b-1225-GGUF --include "miscii-14b-1225-Q4_K_M.gguf" --local-dir ./
   

   If the model is larger than 50GB, it may be split into multiple files.

3. Select Appropriate Quantization:

   • For maximum speed, fit the model into your GPU's VRAM.
   • For maximum quality, consider the combined RAM and VRAM.

4. Run the Model: Use a platform like LM Studio for execution.

Cloud GPU Suggestion: Consider using cloud services like AWS or GCP with NVIDIA GPUs for efficient model execution.

License

MISCII-14B-1225-GGUF is released under the Apache 2.0 license, which allows for both personal and commercial use, modification, and distribution of the software.