miscii 14b 1225 i1 G G U F LLM Model

Introduction

MISCII-14B-1225-I1-GGUF is a model repository by Mradermacher on Hugging Face, featuring weighted/imatrix quantizations of the original model, sthenno-com/miscii-14b-1225. It supports both English and Chinese languages and is built on the Transformers library. The repository provides static and IQ quantized versions of the model, optimized for various size and quality requirements.

Architecture

The model architecture is based on the Transformers library, with quantizations performed by Mradermacher. The repository includes a diverse range of quant types, sorted by size, allowing users to select based on their specific requirements for performance and quality.

Training

The quantizations were made possible through the resources provided by nethype GmbH and a private supercomputer, as acknowledged by Mradermacher. These resources allowed for higher quality imatrix quantizations than could be achieved otherwise.

Guide: Running Locally

Download the Model: Visit the model's Hugging Face page and select the desired GGUF quant file to download.
Install Dependencies: Ensure you have the transformers library installed. You can do this via pip:
```
pip install transformers
```

Load the Model: Use the Transformers library to load and run the model locally:

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("mradermacher/miscii-14b-1225-i1-GGUF")
model = AutoModelForCausalLM.from_pretrained("mradermacher/miscii-14b-1225-i1-GGUF")

input_text = "Your text here"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)

Cloud GPUs: For optimal performance, especially with larger models, consider using a cloud GPU service such as AWS EC2, Google Cloud Platform, or Azure to handle the computational demands.

License

The model and its quantizations are provided under the Apache-2.0 license, allowing for both personal and commercial use, with appropriate attributions.

More Related APIs