Control Nanuq 8 B G G U F

Introduction

Control-Nanuq-8B-GGUF is a quantized version of the Delta-Vector/Control-Nanuq-8B model, designed for a variety of applications such as chat, roleplay, and storywriting. The model is fine-tuned on LLaMA 3.1 8B Supernova and emphasizes concise, coherent output.

Architecture

Control-Nanuq-8B-GGUF employs the LLaMA-Instruct formatting for input prompts, designed to enhance interaction and narrative capabilities. The model includes features to support character embodiment and narrative consistency, adhering to specific role-playing guidelines.

Training

The model was fine-tuned over four epochs using datasets like OpenCAI and RP logs, with techniques such as DPO and KTO reinforcement learning. The training utilized multiple NVIDIA GPUs, including RTX 3090s and a T4 for DPO tuning, with an H100 GPU for KTO training.

Guide: Running Locally

1. Clone the Repository: Start by cloning the model repository from Hugging Face.
2. Set Up Environment: Install necessary dependencies, ensuring compatibility with your system's configuration.
3. Download Model: Use Hugging Face's transformers library to download the model weights.
4. Run Inference: Implement a script to run the model on your input data.
5. Resources: For enhanced performance, consider using cloud GPUs like NVIDIA T4 or H100 available on platforms like AWS, GCP, or Azure.

License

The model and associated content are made available under specific licensing agreements. Refer to the Hugging Face model card for detailed licensing information.