ru T5 base

Introduction

The RUT5-BASE model is a pre-trained Transformer language model designed for the Russian language. It is developed by SberDevices and is focused on text-to-text generation tasks.

Architecture

• Task: Text-to-text generation.
• Type: Encoder-decoder.
• Tokenizer: Byte Pair Encoding (BPE).
• Dictionary Size: 32,101 tokens.
• Number of Parameters: 222 million.

Training

The model was trained on a substantial dataset with a total volume of 300 GB. The training details and evaluation metrics are documented in the preprint titled "A Family of Pretrained Transformer Language Models for Russian" available on arXiv.

Guide: Running Locally

1. Setup Environment: Install the necessary libraries, primarily PyTorch and Hugging Face Transformers.
2. Download Model: Access the model via the Hugging Face Model Hub.
3. Run Inference: Use the model for text-to-text generation tasks by utilizing PyTorch and Transformers.
4. Cloud GPUs: For enhanced performance and faster inference, consider using cloud GPU services like AWS EC2, Google Cloud, or Azure.

License

The model and its resources are available on the Hugging Face platform, with usage subject to the platform's licensing agreements. Ensure to review specific licensing details for compliance.