msmarco Mini L M L6 cos v5

Introduction

The MSMARCO-MiniLM-L6-Cos-V5 is a model from the Sentence Transformers library that maps sentences and paragraphs to a 384-dimensional dense vector space, tailored for semantic search. It was trained using 500,000 (query, answer) pairs from the MS MARCO Passages dataset. This model is particularly useful for applications involving sentence similarity and feature extraction.

Architecture

• Dimensions: 384
• Produces Normalized Embeddings: Yes
• Pooling Method: Mean pooling
• Suitable Score Functions: Dot-product, cosine-similarity, or Euclidean distance. When using the Sentence Transformers library, the model produces normalized embeddings, making dot-product and cosine-similarity equivalent, with dot-product being preferred for its speed.

Training

The model was trained by the Sentence Transformers team, leveraging the MS MARCO Passages dataset. It applies mean pooling over token embeddings to generate sentence embeddings. The publication detailing the methodology is titled "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks."

Guide: Running Locally

To use the MSMARCO-MiniLM-L6-Cos-V5 model locally, follow these steps:

1. Install Dependencies:

   • Install the Sentence Transformers library:

     pip install -U sentence-transformers
     

2. Load and Use the Model:

   • Via Sentence Transformers:

     from sentence_transformers import SentenceTransformer, util
     model = SentenceTransformer('sentence-transformers/msmarco-MiniLM-L6-cos-v5')
     # Encode text and compute scores as needed
     

   • Via Hugging Face Transformers:

     from transformers import AutoTokenizer, AutoModel
     tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/msmarco-MiniLM-L6-cos-v5")
     model = AutoModel.from_pretrained("sentence-transformers/msmarco-MiniLM-L6-cos-v5")
     # Encode text and compute scores using pooling functions
     

3. Compute Scores:

   • Use dot-product or cosine similarity to assess sentence similarity.

Cloud GPUs

For enhanced performance, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure. These platforms offer scalable resources that can significantly speed up the computation needed for training and inference.

License

The model and its associated code are available under the license specified by the Sentence Transformers library. For detailed licensing information, refer to the official Sentence Transformers documentation or the Hugging Face model card.