PEGASUS-NEWSROOM-REWRITER

Introduction

PEGASUS-NEWSROOM-REWRITER is a fine-tuned version of Google's PEGASUS model, specifically adapted for text summarization tasks. It exhibits notable performance on an unspecified dataset, indicated by various ROUGE metrics.

Architecture

The model builds upon Google's PEGASUS architecture, optimized for abstractive text summarization. It utilizes Transformers and the PyTorch library for its implementation.

Training

Training Hyperparameters

• Learning Rate: 2e-05
• Train Batch Size: 1
• Eval Batch Size: 1
• Seed: 42
• Optimizer: Adam (betas=(0.9, 0.999), epsilon=1e-08)
• LR Scheduler Type: Linear
• Number of Epochs: 4
• Mixed Precision Training: Native AMP

Training Results

• Final Loss: 1.3424
• ROUGE Metrics:
  • Rouge1: 46.6856
  • Rouge2: 31.6377
  • Rougel: 33.2741
  • Rougelsum: 44.5003
• Generated Length: 126.58

Framework Versions

• Transformers: 4.17.0
• PyTorch: 1.10.0+cu111
• Datasets: 2.0.0
• Tokenizers: 0.11.6

Guide: Running Locally

1. Clone the Repository:

   git clone https://huggingface.co/chinhon/pegasus-newsroom-rewriter
   cd pegasus-newsroom-rewriter
   

2. Install Dependencies:

   pip install transformers torch datasets
   

3. Load and Use the Model:

   from transformers import PegasusForConditionalGeneration, PegasusTokenizer
   
   model_name = "chinhon/pegasus-newsroom-rewriter"
   tokenizer = PegasusTokenizer.from_pretrained(model_name)
   model = PegasusForConditionalGeneration.from_pretrained(model_name)
   
   text = "Your text here."
   inputs = tokenizer(text, return_tensors='pt', truncation=True, padding='longest')
   summary_ids = model.generate(inputs['input_ids'])
   summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
   print(summary)
   

4. Hardware Recommendations: For optimal performance, it is recommended to use cloud GPU services such as Google Colab, AWS, or Azure.

License

The model is shared under the Apache 2.0 License, which allows for both personal and commercial use, distribution, and modification. Users must ensure compliance with the license terms when using the model.