multi qa Mini L M L6 dot v1

Introduction

The Multi-QA-MiniLM-L6-Dot-V1 is a sentence-transformers model designed for semantic search. It maps sentences and paragraphs to a 384-dimensional dense vector space and has been trained on a diverse set of 215 million question-answer pairs.

Architecture

The model is based on a pre-trained nreimers/MiniLM-L6-H384-uncased architecture. It uses CLS pooling for encoding inputs into dense vectors, with dot-product as the primary similarity function.

Training

The training process utilized diverse datasets, including WikiAnswers, Stack Exchange, MS MARCO, and others, to fine-tune the model. The MultipleNegativesRankingLoss was used to optimize the model with CLS pooling and dot-product similarity. The model was trained on 7 TPUs v3-8 during a community event organized by Hugging Face.

Guide: Running Locally

To use the model locally:

1. Install Dependencies:

   pip install -U sentence-transformers
   

2. Load and Use the Model:

   from sentence_transformers import SentenceTransformer, util
   
   model = SentenceTransformer('sentence-transformers/multi-qa-MiniLM-L6-dot-v1')
   query = "How many people live in London?"
   docs = ["Around 9 Million people live in London", "London is known for its financial district"]
   query_emb = model.encode(query)
   doc_emb = model.encode(docs)
   scores = util.dot_score(query_emb, doc_emb)[0].cpu().tolist()
   

3. Consider Cloud GPUs: For large-scale inference or training, consider using cloud GPUs from providers like AWS, GCP, or Azure to expedite computations.

License

The model is distributed under the terms specified by Hugging Face. For detailed licensing information, please refer to the official Hugging Face website or the model card.