t5 base indonesian summarization cased

Introduction

The t5-base-indonesian-summarization-cased model is a fine-tuned version of the T5 base model designed for text summarization in Indonesian. This model has been adapted from the t5-base-bahasa-summarization-cased by Husein Zolkepli and further fine-tuned using the id_liputan6 dataset.

Architecture

This model is based on the T5 (Text-to-Text Transfer Transformer) architecture, which is a versatile model designed for various NLP tasks by transforming all tasks into a text-to-text format. The model uses the pre-trained T5 architecture and has been specifically tailored for summarization tasks in the Indonesian language.

Training

The model was fine-tuned using the id_liputan6 dataset, which is a collection of Indonesian news articles. The fine-tuning process involves adapting the T5 base model weights to perform effectively on the summarization task specific to this dataset.

Guide: Running Locally

To run the model locally, you can use the following Python code sample:

1. Install the Transformers library if not already installed:

   pip install transformers
   

2. Load the model and tokenizer:

   from transformers import T5Tokenizer, T5ForConditionalGeneration
   
   tokenizer = T5Tokenizer.from_pretrained("cahya/t5-base-indonesian-summarization-cased")
   model = T5ForConditionalGeneration.from_pretrained("cahya/t5-base-indonesian-summarization-cased")
   

3. Prepare the text and generate a summary:

   ARTICLE_TO_SUMMARIZE = "Your article text here."
   
   input_ids = tokenizer.encode(ARTICLE_TO_SUMMARIZE, return_tensors='pt')
   summary_ids = model.generate(input_ids,
               min_length=20,
               max_length=80,
               num_beams=10,
               repetition_penalty=2.5,
               length_penalty=1.0,
               early_stopping=True,
               no_repeat_ngram_size=2,
               use_cache=True,
               do_sample=True,
               temperature=0.8,
               top_k=50,
               top_p=0.95)
   
   summary_text = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
   print(summary_text)
   

4. Suggest using cloud GPUs for better performance, especially for large datasets or intensive tasks. Providers like AWS, Google Cloud, or Azure offer suitable options.

License

The model is hosted on Hugging Face and may be subject to the licensing terms applicable to the T5 model and the id_liputan6 dataset. Users should refer to the respective licenses for detailed information.