gec t5_small

Introduction
The GEC-T5_SMALL model is designed for Grammatical Error Correction (GEC) using a T5-small architecture. It achieves state-of-the-art scores as reported in the paper "A Simple Recipe for Multilingual Grammatical Error Correction" by Google, with an F_0.5 score of 60.70. The model is suitable for text-to-text generation tasks, particularly focused on correcting grammatical errors in English sentences.

Architecture
The model uses the T5-small transformer architecture. It is implemented in PyTorch and is compatible with the Hugging Face Transformers library. The model is trained and evaluated using datasets such as clang-8, conll-14, and conll-13.

Training
The model has been optimized for GEC tasks, receiving training on large datasets to enhance its capability to correct grammatical errors. The use of an F_0.5 score during training allows the model to balance precision and recall, prioritizing grammatical accuracy while maintaining fluency.

Guide: Running Locally
To use the GEC-T5_SMALL model locally, you can follow these steps:

1. Install the Hugging Face Transformers library:

   pip install transformers
   

2. Use the following Python code snippet to load the model and tokenizer:

   from transformers import T5ForConditionalGeneration, T5Tokenizer
   
   model = T5ForConditionalGeneration.from_pretrained("Unbabel/gec-t5_small")
   tokenizer = T5Tokenizer.from_pretrained('t5-small')
   
   sentence = "I like to swimming"
   tokenized_sentence = tokenizer('gec: ' + sentence, max_length=128, truncation=True, padding='max_length', return_tensors='pt')
   corrected_sentence = tokenizer.decode(
       model.generate(
           input_ids = tokenized_sentence.input_ids,
           attention_mask = tokenized_sentence.attention_mask, 
           max_length=128,
           num_beams=5,
           early_stopping=True,
       )[0],
       skip_special_tokens=True, 
       clean_up_tokenization_spaces=True
   )
   print(corrected_sentence) # -> I like swimming.
   

3. For better performance, especially when processing larger datasets or running multiple instances, consider using cloud-based GPUs such as those provided by AWS, Google Cloud, or Azure.

License
The GEC-T5_SMALL model is released under the Apache-2.0 license, allowing for both commercial and non-commercial use, modification, and distribution.