Dans Personality Engine V1.1.0 12b G G U F

Introduction

Dan's Personality Engine V1.1.0-12B-GGUF is a versatile text generation model designed for various applications, including roleplay, story writing, and scientific domains like chemistry and biology. It is built on the GGUF framework and supports fine-tuning for specific tasks and conversational scenarios.

Architecture

The model is based on the PocketDoc/Dans-PersonalityEngine-V1.1.0-12b architecture. It uses sophisticated quantization methods, including IMATRIX via the llama.cpp platform, to offer various quantization levels from high-quality (Q8_0) to lower-quality options (IQ2_M), tailored for different hardware capabilities and performance requirements.

Training

The model is trained on a diverse set of datasets, including those focused on mathematics, code, survival, and various scientific topics. This wide range of data sources allows the model to handle multiple domains effectively. The quantization process includes embedding and output weights optimization, enhancing the model’s efficiency and versatility.

Guide: Running Locally

1. Set Up Environment:

   • Ensure you have Python and huggingface_hub installed:

     pip install -U "huggingface_hub[cli]"
     

2. Download Model:

   • Use the huggingface-cli to download specific model files:

     huggingface-cli download bartowski/Dans-PersonalityEngine-V1.1.0-12b-GGUF --include "Dans-PersonalityEngine-V1.1.0-12b-Q4_K_M.gguf" --local-dir ./
     

   • For models larger than 50GB, specify the directory for all parts:

     huggingface-cli download bartowski/Dans-PersonalityEngine-V1.1.0-12b-GGUF --include "Dans-PersonalityEngine-V1.1.0-12b-Q8_0/*" --local-dir ./
     

3. Run the Model:

   • Consider using platforms like LM Studio for running the model efficiently.
   • Choose an appropriate quant file based on your hardware's RAM or VRAM capabilities. For optimal performance, select a quant size smaller than your available VRAM by 1-2GB.

4. Cloud GPUs:

   • For large models and faster processing, consider running the model on cloud-based GPUs such as AWS, Google Cloud, or Azure.

License

Dan's Personality Engine V1.1.0-12B-GGUF is released under the Apache-2.0 license, which allows for both personal and commercial use with proper attribution.