rebel large

Introduction

REBEL (Relation Extraction By End-to-end Language generation) is a model designed to extract relation triplets from text using a seq2seq (sequence-to-sequence) approach. It leverages the BART model architecture for end-to-end relation extraction, handling over 200 different relation types.

Architecture

REBEL reframes the Relation Extraction task as a seq2seq task, utilizing BART, a transformer-based model, to process and generate relation triplets. This approach simplifies traditional multi-step pipelines by directly generating triplets as text sequences.

Training

REBEL is fine-tuned on several benchmarks for Relation Extraction and Relation Classification, achieving state-of-the-art performance in many instances. The model is trained using datasets such as CoNLL04 and NYT, with performance metrics like F1 scores reported for these benchmarks.

Guide: Running Locally

1. Install the Transformers Library: Ensure you have the transformers library installed.

   pip install transformers
   

2. Load the Model and Tokenizer:

   from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
   
   tokenizer = AutoTokenizer.from_pretrained("Babelscape/rebel-large")
   model = AutoModelForSeq2SeqLM.from_pretrained("Babelscape/rebel-large")
   

3. Prepare the Input Text:

   text = 'Punta Cana is a resort town in the municipality of Higüey, in La Altagracia Province, the easternmost province of the Dominican Republic.'
   model_inputs = tokenizer(text, max_length=256, padding=True, truncation=True, return_tensors='pt')
   

4. Generate the Output:

   gen_kwargs = {
       "max_length": 256,
       "length_penalty": 0,
       "num_beams": 3,
       "num_return_sequences": 3,
   }
   generated_tokens = model.generate(
       model_inputs["input_ids"].to(model.device),
       attention_mask=model_inputs["attention_mask"].to(model.device),
       **gen_kwargs,
   )
   

5. Decode and Extract Triplets:

   decoded_preds = tokenizer.batch_decode(generated_tokens, skip_special_tokens=False)
   # Function to parse the generated text and extract the triplets
   def extract_triplets(text):
       # Implement the extraction logic as shown in the document
   for idx, sentence in enumerate(decoded_preds):
       print(f'Prediction triplets sentence {idx}')
       print(extract_triplets(sentence))
   

6. Cloud GPUs: For faster processing, consider using cloud GPUs available on platforms like AWS, Google Cloud, or Azure.

License

The REBEL model is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (cc-by-nc-sa-4.0). This license allows for sharing and adapting the model, provided that appropriate credit is given, and the use is non-commercial.