multi qa mpnet base dot v1

Introduction

The Multi-QA-MPNET-Base-Dot-V1 is a sentence-transformers model designed for semantic search. It maps sentences and paragraphs to a 768-dimensional dense vector space and is trained on 215M question-answer pairs from various sources. The model facilitates efficient semantic search by encoding queries and documents, allowing for the retrieval of relevant documents based on semantic similarity.

Architecture

The model is based on the MPNet architecture, which is known for its efficiency in understanding sentence context. It leverages the CLS pooling method and uses dot-product similarity as the scoring function. The model produces 768-dimensional embeddings suitable for sentence similarity tasks.

Training

The model was trained using a self-supervised contrastive learning objective on a diverse set of datasets, comprising approximately 215 million question-answer pairs. Key datasets include WikiAnswers, PAQ, Stack Exchange, MS MARCO, and several others. The training employed the MultipleNegativesRankingLoss function with CLS pooling and dot-product scoring. The training process utilized advanced hardware, including TPUs, and was part of a community-driven project using JAX/Flax for NLP.

Guide: Running Locally

Basic Steps

1. Install Dependencies:

   pip install -U sentence-transformers
   

2. Load and Use the Model:

   from sentence_transformers import SentenceTransformer, util
   
   model = SentenceTransformer('sentence-transformers/multi-qa-mpnet-base-dot-v1')
   

3. Encode Sentences:

   query = "How many people live in London?"
   docs = ["Around 9 Million people live in London", "London is known for its financial district"]
   query_emb = model.encode(query)
   doc_emb = model.encode(docs)
   

4. Compute Similarity Scores:

   scores = util.dot_score(query_emb, doc_emb)[0].cpu().tolist()
   

5. Sort and Display Results:

   doc_score_pairs = list(zip(docs, scores))
   doc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
   for doc, score in doc_score_pairs:
       print(score, doc)
   

Cloud GPUs

For heavy workloads or large datasets, using a cloud GPU service like AWS, Google Cloud, or Microsoft Azure is recommended. These platforms provide scalable solutions that can handle extensive computations required by the model.

License

The model and code are available under the Apache 2.0 license, allowing for both personal and commercial use. Please refer to the license file in the repository for detailed terms and conditions.