Qwen2.5 32b R P Ink

Introduction

QWEN 2.5 32B RP INK is a roleplay-focused, LoRA-finetuned version of the Qwen 2.5 32B Instruct model. It draws inspiration from SorcererLM and Slush methodologies. The model is a part of the Ink series, known for its engaging prose and accurate character portrayal.

Architecture

QWEN 2.5 32B RP INK is based on the Qwen/Qwen2.5-32B-Instruct model. It is designed to handle complex scenarios effectively, even outperforming some larger models in terms of character portrayal and scene setting.

Training

The training methodology involves:

• Epochs: 1
• Learning Rate: 6e-5
• Scheduler: Cosine
• Optimizer: Paged AdamW 8bit
• Batch Size: 16

LoRA-specific parameters include:

• Rank: 16
• Alpha: 32
• Dropout: 0.25

Guide: Running Locally

To run QWEN 2.5 32B RP INK locally, follow these steps:

1. Clone the repository: Obtain the model files from the Hugging Face repository.
2. Install dependencies: Ensure you have the necessary packages installed, such as PyTorch and Hugging Face Transformers.
3. Load the model: Use the Hugging Face library to load the model into your environment.
4. Configure: Set your preferred chat template and sampler settings, like Temp 0.85 / Top P 0.8.
5. Run: Execute your scripts to interact with the model.

For optimal performance, consider using a cloud GPU service like AWS, Google Cloud, or Azure.

License

The model is licensed under the Apache-2.0 License, allowing for both personal and commercial use with proper attribution.