rut5 base absum

Introduction

The rut5-base-absum model is designed for abstractive summarization in the Russian language. It is based on the cointegrated/rut5-base-multitask model and fine-tuned on four datasets. This model is focused on generating concise summaries from longer Russian texts.

Architecture

The model architecture is derived from the T5 (Text-to-Text Transfer Transformer) framework, utilizing PyTorch for implementation. It supports text-to-text generation tasks, specifically targeting summarization in Russian.

Training

The model is fine-tuned on four datasets: IlyaGusev/gazeta, csebuetnlp/xlsum, mlsum, and wiki_lingua. This training provides the model with a robust foundation for generating high-quality summaries in the Russian language.

Guide: Running Locally

To run the rut5-base-absum model locally, follow these steps:

1. Install Required Libraries: Ensure that transformers and torch are installed in your Python environment.

   pip install transformers torch
   

2. Load the Model: Use the following code snippet to load and prepare the model.

   import torch
   from transformers import T5ForConditionalGeneration, T5Tokenizer
   
   MODEL_NAME = 'cointegrated/rut5-base-absum'
   model = T5ForConditionalGeneration.from_pretrained(MODEL_NAME)
   tokenizer = T5Tokenizer.from_pretrained(MODEL_NAME)
   model.cuda()
   model.eval()
   

3. Summarize Text: Utilize the summarize function to generate summaries.

   def summarize(text, n_words=None, compression=None, max_length=1000, num_beams=3, do_sample=False, repetition_penalty=10.0, **kwargs):
       if n_words:
           text = '[{}] '.format(n_words) + text
       elif compression:
           text = '[{0:.1g}] '.format(compression) + text
       x = tokenizer(text, return_tensors='pt', padding=True).to(model.device)
       with torch.inference_mode():
           out = model.generate(**x, max_length=max_length, num_beams=num_beams, do_sample=do_sample, repetition_penalty=repetition_penalty, **kwargs)
       return tokenizer.decode(out[0], skip_special_tokens=True)
   

4. Example Usage:

   text = "Высота башни составляет 324 метра..."
   print(summarize(text))
   

Suggestion for Cloud GPUs: Consider using cloud-based services like AWS, Google Cloud, or Azure for GPU acceleration if running on local hardware is insufficient.

License

The rut5-base-absum model is licensed under the MIT License, allowing for wide usage and modification with proper attribution.