miscii 14b 1028 G G U F

Introduction

The MISCII-14B-1028-GGUF is a quantized version of the model originally developed by Sthenno. It supports text generation tasks and is designed for conversational and roleplay applications. The model is built using the Transformers library and is optimized for performance using the GGUF library.

Architecture

MISCII-14B-1028-GGUF is based on the Qwen/Qwen2.5-14B-Instruct model. It is tailored for tasks involving chat, conversational, custom research, and roleplay. The model employs role-based instructions to guide interactions, utilizing a system prompt that defines user and assistant personas.

Training

The model underwent training with various datasets, including nvidia/HelpSteer2, google/Synthetic-Persona-Chat, and mlabonne/orpo-dpo-mix-40k. Performance metrics from the Open LLM Leaderboard indicate its capabilities across multiple datasets, achieving an 82.37% strict accuracy on IFEval (0-Shot) and a 46.14% accuracy on MMLU-PRO (5-shot), among others.

Guide: Running Locally

1. Setup Environment: Ensure you have a Python environment with the Transformers library installed.
2. Download Model: Clone the model repository from Hugging Face.
3. Load Model: Use the Transformers library to load the model and tokenizer.
4. Run Inference: Prepare input prompts according to the role-based instruction format and execute text generation tasks.

For optimal performance, consider using cloud GPUs such as those provided by AWS, Google Cloud, or Azure, especially when handling large datasets or complex computations.

License

The model is licensed under the Apache-2.0 License, allowing for wide usage and modification with proper attribution.