Bielik 11 B v2.3 Instruct G G U F

Introduction

The Bielik-11B-v2.3-Instruct-GGUF model is a quantized text generation model developed by SpeakLeash in collaboration with ACK Cyfronet AGH. It is optimized for the Polish language and is available under the Apache 2.0 license.

Architecture

This model is a causal, decoder-only model derived from Bielik-11B-v2.3-Instruct and finetuned from Bielik-11B-v2. It uses the GGUF format, introduced by the llama.cpp team, which supports various quantization levels for efficient performance.

Training

The model has undergone quantization to reduce resource usage while maintaining performance. Supported quantization formats include q4_k_m, q5_k_m, q6_k, and q8_0, each offering different trade-offs between resource use and output quality.

Guide: Running Locally

To run the Bielik-11B-v2.3-Instruct-GGUF model locally, you can use several compatible clients and libraries that support the GGUF format:

1. Install Required Software:

   • Choose a compatible library or client, such as llama.cpp, text-generation-webui, or GPT4All.

2. Download Model Files:

   • Obtain the model in the GGUF format from the Hugging Face repository.

3. Configure the Modfile:

   • For example, for Ollama, set the path to the model and configure the parameters as needed.

4. Run the Model:

   • Use a cloud GPU service, such as AWS or Google Cloud, for enhanced performance.

License

The Bielik-11B-v2.3-Instruct-GGUF model is licensed under the Apache 2.0 license, which allows for wide usage and modification with attribution. For detailed terms of use, visit Bielik.ai Terms of Use.