Teleut 7b R P

Introduction

Teleut-7B-RP is a roleplay-focused LoRA fine-tuned model based on Teleut 7B. The methodology and hyperparameters are inspired by SorcererLM and Slush models. This model is designed for conversational and roleplay applications.

Architecture

This model is a fine-tuned version of the Teleut 7B model, optimized for roleplay and conversational use cases. The architecture leverages LoRA techniques to enhance specific functionalities.

Training

Dataset

The model is trained on a diverse and unconventional dataset, humorously noted for its poor quality.

Hyperparameters

• Epochs: 2
• Learning Rate (LR): 6e-5
• LR Scheduler: Cosine
• Optimizer: Paged AdamW 8bit
• Effective Batch Size: 12

LoRA Specifics

• Rank: 16
• Alpha: 32
• Dropout: 0.25

Guide: Running Locally

To run Teleut-7B-RP locally, follow these steps:

1. Clone the Repository: Clone the model repository from Hugging Face.
2. Install Dependencies: Ensure all required libraries and dependencies are installed.
3. Download Model Weights: Obtain the model weights and configurations from the repository.
4. Run Model: Initiate the model using a preferred machine learning framework.

Cloud GPUs

For optimal performance, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

This model is released under the Apache 2.0 license, allowing for broad use, modification, and distribution with proper attribution.