Peach 9 B 8k Roleplay

Introduction
Peach-9B-8k-Roleplay is a chat-oriented large language model fine-tuned from the 01-ai/Yi-1.5-9B model. It has been trained on over 100,000 synthesized conversations, providing robust roleplay capabilities in both English and Chinese languages. It is considered one of the best models with a small parameter count under 34 billion.

Architecture
The model is based on a large language model architecture optimized for roleplay and conversational tasks. It supports text generation in both English and Chinese, utilizing a dataset designed to enhance its capabilities in character dialogues and roleplaying scenarios.

Training
Peach-9B-8k-Roleplay was fine-tuned using data synthesis techniques to create a diverse set of over 100K conversations. This approach aims to enhance the model's ability to generate engaging and contextually appropriate roleplay dialogue. It is bilingual, trained in both English and Chinese, though it may not perform as well in low-resource languages.

Guide: Running Locally
To run Peach-9B-8k-Roleplay locally, ensure you have the following Python packages installed:

• torch==1.13.1
• gradio==3.50.2
• transformers==4.37.2

Use the following Python code for inference:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name_or_path = "ClosedCharacter/Peach-9B-8k-Roleplay"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
    model_name_or_path, torch_dtype=torch.bfloat16, 
    trust_remote_code=True, device_map="auto")

messages = [
    {"role": "system", "content": "你是黑丝御姐"},
    {"role": "user", "content": "你好，你是谁"},
]

input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, return_tensors="pt")
output = model.generate(
    inputs=input_ids.to("cuda"), 
    temperature=0.3, 
    top_p=0.5, 
    no_repeat_ngram_size=6,
    repetition_penalty=1.1,
    max_new_tokens=512)

print(tokenizer.decode(output[0]))

Alternatively, run the demo using:

python demo.py

For optimal performance, consider using a cloud GPU service to handle the model's resource requirements.

License
Peach-9B-8k-Roleplay is licensed under the MIT License, allowing for flexible use and modification.