vit5 base vietnews summarization
VietAIIntroduction
The ViT5-BASE-VIETNEWS-SUMMARIZATION model is a state-of-the-art Transformer-based encoder-decoder model specifically designed for abstractive text summarization in Vietnamese. It has been fine-tuned on the VietNews dataset to optimize its performance in summarizing Vietnamese news articles.
Architecture
ViT5 is built on the Transformer architecture, leveraging both encoder and decoder components to process inputs and generate summaries. It uses a pretrained text-to-text approach, which has been adapted for the Vietnamese language to ensure high-quality summarization tasks.
Training
The model is trained using the CC100 dataset, which provides a diverse and extensive collection of Vietnamese text. This training setup allows the model to effectively learn linguistic nuances and context, resulting in accurate and coherent summaries.
Guide: Running Locally
To run the ViT5 model locally, follow these steps:
-
Install Transformers Library: Ensure you have the
transformers
library installed in your Python environment. -
Load the Model: Use the following code to load the tokenizer and model:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("VietAI/vit5-base-vietnews-summarization") model = AutoModelForSeq2SeqLM.from_pretrained("VietAI/vit5-base-vietnews-summarization") model.cuda()
-
Prepare Input Text: Format your input text and tokenize it.
sentence = "Your Vietnamese text here." sentence = sentence + "</s>" encoding = tokenizer(sentence, return_tensors="pt") input_ids, attention_masks = encoding["input_ids"].to("cuda"), encoding["attention_mask"].to("cuda")
-
Generate Summary: Pass the tokenized input to the model and decode the output.
outputs = model.generate( input_ids=input_ids, attention_mask=attention_masks, max_length=256, early_stopping=True ) for output in outputs: line = tokenizer.decode(output, skip_special_tokens=True, clean_up_tokenization_spaces=True) print(line)
-
Cloud GPUs: To optimize performance, consider using cloud-based GPUs such as those available on AWS, Google Cloud, or Azure.
License
The ViT5-BASE-VIETNEWS-SUMMARIZATION model is released under the MIT License, which allows for flexibility in usage and distribution.