legal led base 16384

Introduction

The legal-led-base-16384 model is a Longformer Encoder Decoder (LED) model designed specifically for abstractive summarization in the legal domain. It can process documents up to 16,384 tokens in length.

Architecture

This model is based on the Longformer Encoder Decoder (LED) architecture, which is particularly suited for handling long documents. The base model, led-base-16384, serves as the foundation, enabling efficient summarization tasks by accommodating extended input sequences typical of legal documents.

Training

The model was trained on the SEC litigation releases dataset, which includes more than 2700 litigation releases and complaints. This extensive dataset helps the model to effectively summarize complex legal texts.

Guide: Running Locally

To use the model locally, follow these steps:

1. Install Dependencies: Ensure you have the transformers library installed.

   pip install transformers
   

2. Load the Model and Tokenizer:

   from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
   
   tokenizer = AutoTokenizer.from_pretrained("nsi319/legal-led-base-16384")
   model = AutoModelForSeq2SeqLM.from_pretrained("nsi319/legal-led-base-16384")
   

3. Prepare Input Text: Tokenize and prepare your document for summarization.
4. Generate Summary:

   input_tokenized = tokenizer.encode(text, return_tensors='pt', padding='max_length', pad_to_max_length=True, max_length=6144, truncation=True)
   summary_ids = model.generate(input_tokenized, num_beams=4, no_repeat_ngram_size=3, length_penalty=2, min_length=350, max_length=500)
   summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
   

For optimal performance and efficiency, especially with large documents, consider using cloud-based GPUs such as those provided by AWS, Google Cloud, or Azure.

License

The legal-led-base-16384 model is licensed under the MIT License, allowing for wide usage and modification under specified terms.