rugpt3_song_writer

Introduction

The rugpt3_song_writer is a model designed for generating song lyrics in Russian. It is based on the ruGPT-3 model and has been fine-tuned using lyrics sourced from Genius. The model can emulate styles of various Russian artists, such as Oxxxymiron, Моргенштерн, ЛСП, and others.

Architecture

The model is built on the ruGPT-3 architecture, which is a variant of GPT-2, specifically adapted for the Russian language. It leverages the Transformers library and operates within the PyTorch framework, allowing for efficient text generation tasks.

Training

The model was fine-tuned on a dataset of song lyrics collected from Genius. This dataset includes a variety of artists, providing a diverse range of lyrical styles for the model to learn from. The model's hyperparameters for inference include a temperature of 0.9, a top-k of 50, a top-p of 0.95, and a maximum length of 1500 tokens.

Guide: Running Locally

To run the rugpt3_song_writer model locally, follow these steps:

1. Install the Transformers and PyTorch libraries:

   pip install transformers torch
   

2. Load the model:

   from transformers import GPT2LMHeadModel, GPT2Tokenizer
   
   model_name = "bankholdup/rugpt3_song_writer"
   model = GPT2LMHeadModel.from_pretrained(model_name)
   tokenizer = GPT2Tokenizer.from_pretrained(model_name)
   

3. Generate text:

   input_text = "Ваш текст здесь"
   input_ids = tokenizer.encode(input_text, return_tensors='pt')
   output = model.generate(input_ids, max_length=1500, temperature=0.9, top_k=50, top_p=0.95)
   print(tokenizer.decode(output[0], skip_special_tokens=True))
   

For better performance, consider using a cloud GPU service like AWS, Google Cloud, or Azure.

License

The model is shared under the terms specified by Hugging Face and the original creators. Users should review these terms to ensure compliance with any usage restrictions or requirements.