F L A N T5 Paraphraser

Introduction

The FLAN-T5-Paraphraser is a model designed for generating high-quality paraphrases, emphasizing fluency, diversity, and relevance. It is particularly useful for adversarial data generation, aiding in the discovery of edge cases with minimal distribution distortion. This model was developed as part of the research presented in the paper titled "Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion."

Architecture

The model is based on the FLAN-T5-large architecture. It was fine-tuned on a carefully curated dataset to enhance its ability to generate diverse and fluent paraphrases. The training process focused on optimizing the model for task-specific outputs, ensuring high-quality paraphrasing.

Training

Training utilized a dataset of 560,550 paraphrase pairs from various high-quality sources, narrowed down to 96,073 samples after rigorous filtering for diversity and semantic similarity. The model was trained for nine epochs, achieving an F1 BERT-Score of 75.925%, indicating its superior paraphrasing capabilities.

Guide: Running Locally

To run the FLAN-T5-Paraphraser locally:

1. Installation: Install the Parrot library using:

   pip install git+https://github.com/PrithivirajDamodaran/Parrot_Paraphraser.git
   

2. Usage with Transformers:

   from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
   
   model_name = "alykassem/FLAN-T5-Paraphraser"
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
   
   input_text = "Paraphrase: How are you?"
   inputs = tokenizer(input_text, return_tensors="pt")
   outputs = model.generate(**inputs)
   decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
   print("Generated text:", decoded_output)
   

3. Usage with Parrot:

   from parrot import Parrot
   import warnings
   warnings.filterwarnings("ignore")
   
   parrot = Parrot(model_tag="alykassem/FLAN-T5-Paraphraser", use_gpu=True)
   
   phrases = [
       "Can you recommend some upscale restaurants in New York?",
       "What are the famous places we should not miss in Russia?"
   ]
   
   for phrase in phrases:
       para_phrases = parrot.augment(input_phrase=phrase)
       for para_phrase in para_phrases:
           print(para_phrase)
   

Cloud GPUs are recommended for faster performance and efficiency when running large models like FLAN-T5.

License

The FLAN-T5-Paraphraser is released under the Apache-2.0 license, allowing for free use and distribution, with the requirement to include the same license in any derivative works.