GTR-T5-LARGE Model Documentation

Introduction

The GTR-T5-LARGE model is a sentence-transformers model designed to map sentences and paragraphs into a 768-dimensional dense vector space, specifically trained for semantic search tasks. It has been converted from a TensorFlow model (gtr-large-1) to PyTorch, maintaining consistent performance across benchmarks despite potential differences in embeddings.

Architecture

The model leverages only the encoder from a T5-large model. The weights are stored in FP16 precision, optimizing for performance and efficiency. It is part of the sentence-transformers library, facilitating sentence similarity and feature extraction tasks.

Training

The model was trained as a large dual encoder, which is known for its generalizability in retrieving semantic content. More details can be found in the publication titled "Large Dual Encoders Are Generalizable Retrievers" (arXiv:2112.07899).

Guide: Running Locally

To use the GTR-T5-LARGE model locally, follow these basic steps:

Install the Required Library
Ensure you have the sentence-transformers library version 2.2.0 or newer installed:
```
pip install -U sentence-transformers
```

Using the Model
You can load and use the model as shown below:

from sentence_transformers import SentenceTransformer

sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('sentence-transformers/gtr-t5-large')
embeddings = model.encode(sentences)
print(embeddings)

Cloud GPUs
For better performance, consider running the model on cloud platforms that provide GPU support such as AWS, Google Cloud, or Azure.

License

The GTR-T5-LARGE model is released under the Apache 2.0 license, allowing for wide usage and distribution while maintaining open-source compliance.