sentence t5 xxl

Introduction

The Sentence-T5-XXL is a model from the Sentence-Transformers library developed for mapping sentences and paragraphs into a 768-dimensional dense vector space. It excels in sentence similarity tasks, though it may not perform as well for semantic search tasks.

Architecture

The model is a conversion from the Tensorflow model ST5-11B-1 to PyTorch, using only the encoder from a T5-11B model with weights in FP16. Despite potential differences in embeddings between the Tensorflow and PyTorch versions, they produce identical results on benchmarks.

Training

This model has been trained for sentence similarity tasks, and its performance can be evaluated using the Sentence Embeddings Benchmark available at https://seb.sbert.net.

Guide: Running Locally

1. Installation: Ensure you have sentence-transformers version 2.2.0 or newer installed. You can install it using:

   pip install -U sentence-transformers
   

2. Usage: Load and use the model in your Python environment:

   from sentence_transformers import SentenceTransformer
   sentences = ["This is an example sentence", "Each sentence is converted"]
   
   model = SentenceTransformer('sentence-transformers/sentence-t5-xxl')
   embeddings = model.encode(sentences)
   print(embeddings)
   

3. Hardware Requirement: For optimal performance, it is recommended to use cloud GPUs such as AWS EC2 P3 instances or Google Cloud's GPU offerings.

License

The Sentence-T5-XXL model is licensed under the Apache-2.0 License.