Dans Personality Engine V1.1.0 12b G G U F

Introduction

Dans-PersonalityEngine-V1.1.0-12B-GGUF is a multifaceted AI model intended for various applications, including co-writing, roleplay, sentiment analysis, and summarization. This quantized version is based on the original PocketDoc/Dans-PersonalityEngine-V1.1.0-12b, utilizing the llama.cpp library.

Architecture

This model is built on the mistralai/Mistral-Nemo-Base-2407 base model, supporting English language with a context length of 32,768 tokens. It is designed to handle a variety of tasks, from general-purpose to domain-specific applications like chemistry, biology, and code.

Training

The model was fine-tuned for two epochs using a single H200 SXM GPU over 88 hours. The training process utilized a range of datasets, including those focused on memory, math, programming, writing, and logical reasoning, to enhance its general and specialized capabilities.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install Dependencies: Ensure you have Python and the required libraries installed.
2. Clone the Repository: Download the model files from Hugging Face.
3. Set Up the Environment: Configure your environment to support GPU processing.
4. Load the Model: Use libraries like Transformers to load and interact with the model.
5. Run Inference: Execute tasks and evaluate outputs.

For efficient processing, especially for large tasks, consider using cloud-based GPU services such as AWS EC2, Google Cloud Platform, or Azure.

License

The model is licensed under Apache 2.0, allowing for wide use and distribution with minimal restrictions.