M N 12b R P Ink

Introduction

MN-12B-RP-Ink is a roleplay-focused LoRA finetuned model based on Mistral Nemo Instruct. It adopts methodology and hyperparameters inspired by SorcererLM and Slush. The model is uniquely renamed to "Ink" to distinguish it from other roleplay models, using the same dataset as the Teleut RP model.

Architecture

MN-12B-RP-Ink is built as a LoRA finetune of the Mistral Nemo Instruct model. This architecture helps enhance the model's performance in roleplaying and conversational tasks.

Training

The model was trained with the following hyperparameters:

• Epochs: 2
• Learning Rate: 6e-5
• Learning Rate Scheduler: Cosine
• Optimizer: Paged AdamW 8bit
• Effective Batch Size: 12

For the LoRA component:

• Rank: 16
• Alpha: 32
• Dropout: 0.25

The dataset used for training is described humorously as a mix of unconventional data sources, emphasizing its eclectic nature.

Guide: Running Locally

To run MN-12B-RP-Ink locally, follow these basic steps:

1. Clone the Repository: Ensure you have access to the model files by cloning the repository from Hugging Face.
2. Install Dependencies: Make sure you have the necessary libraries installed, such as transformers and torch.
3. Set Up Environment: Prepare your environment to handle large models, ideally using a cloud GPU provider such as AWS, Google Cloud, or Azure.
4. Load the Model: Use the Hugging Face Transformers library to load the model.
5. Configure Settings: Use recommended chat templates and samplers such as:
   • Temp 1.25 / MinP 0.1
   • Temp 1.03 / TopK 200 / MinP 0.05 / TopA 0.2

License

MN-12B-RP-Ink is licensed under the Apache 2.0 License, allowing for wide usage and modification with compliance to the license terms.