pegasus_paraphrase

Introduction

PEGASUS Paraphrase is a model fine-tuned from Google's PEGASUS, designed for paraphrasing tasks. It leverages the text2text generation capabilities of Transformers with PyTorch, supporting English language paraphrasing via a sequence-to-sequence (seq2seq) approach.

Architecture

PEGASUS Paraphrase utilizes the PEGASUS architecture, which is focused on abstractive text summarization and generation. It applies the Transformer model framework, making it capable of generating paraphrased text efficiently by understanding and rewriting input text.

Training

The model has been fine-tuned specifically for paraphrasing tasks. This process involves adjusting the weights of a pre-trained PEGASUS model to improve its performance on tasks that involve generating varied textual outputs while maintaining the original meaning.

Guide: Running Locally

To run the PEGASUS Paraphrase model locally, follow these steps:

1. Environment Setup:

   • Ensure torch is installed to leverage GPU acceleration if available.
   • Install the transformers library from Hugging Face.

2. Load the Model:

   import torch
   from transformers import PegasusForConditionalGeneration, PegasusTokenizer
   
   model_name = 'tuner007/pegasus_paraphrase'
   torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
   tokenizer = PegasusTokenizer.from_pretrained(model_name)
   model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)
   

3. Generate Paraphrases:

   def get_response(input_text, num_return_sequences, num_beams):
       batch = tokenizer([input_text], truncation=True, padding='longest', max_length=60, return_tensors="pt").to(torch_device)
       translated = model.generate(**batch, max_length=60, num_beams=num_beams, num_return_sequences=num_return_sequences, temperature=1.5)
       tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
       return tgt_text
   
   num_beams = 10
   num_return_sequences = 10
   context = "The ultimate test of your knowledge is your capacity to convey it to another."
   get_response(context, num_return_sequences, num_beams)
   

4. Cloud GPUs:

   • For enhanced performance, consider using cloud GPU services such as AWS, Google Cloud Platform, or Microsoft Azure.

License

PEGASUS Paraphrase is licensed under the Apache-2.0 License, which allows for usage, distribution, and modification with proper attribution.