multi qa Mini L M L6 cos v1

Introduction

The multi-qa-MiniLM-L6-cos-v1 is a Sentence Transformers model that maps sentences and paragraphs to a 384-dimensional dense vector space. It is designed for semantic search and has been trained on 215 million question-answer pairs from diverse sources.

Architecture

This model is based on the MiniLM architecture and uses mean pooling to generate sentence embeddings. It produces normalized embeddings that can be used for various similarity calculations, such as dot-product or cosine-similarity.

Training

The training process leverages a self-supervised contrastive learning objective. The model was fine-tuned on a combination of multiple datasets, resulting in approximately 215 million training pairs. The training employed the MultipleNegativesRankingLoss with mean-pooling and used cosine similarity as the similarity function. The training utilized TPUs to efficiently process the data.

Guide: Running Locally

To run the model locally, follow these steps:

1. Install Sentence Transformers:

   pip install -U sentence-transformers
   

2. Load and Use the Model:

   from sentence_transformers import SentenceTransformer, util
   
   # Load the model
   model = SentenceTransformer('sentence-transformers/multi-qa-MiniLM-L6-cos-v1')
   
   # Encode queries and documents
   query = "How many people live in London?"
   docs = ["Around 9 Million people live in London", "London is known for its financial district"]
   query_emb = model.encode(query)
   doc_emb = model.encode(docs)
   
   # Compute dot score
   scores = util.dot_score(query_emb, doc_emb)[0].cpu().tolist()
   
   # Output results
   for doc, score in zip(docs, scores):
       print(score, doc)
   

3. Alternative Models: You can also use the model with PyTorch or TensorFlow via the Hugging Face Transformers library, applying mean pooling to contextualized word embeddings.

4. Cloud GPUs: For faster processing, consider using cloud-based GPU services like AWS, GCP, or Azure.

License

The model is made available under the Apache 2.0 license, which allows for both personal and commercial use, modification, and distribution. Please ensure compliance with this license in your use case.