multi qa mpnet base cos v1

Introduction

The multi-qa-mpnet-base-cos-v1 model is a sentence-transformers model designed for semantic search. It maps sentences and paragraphs to a 768-dimensional dense vector space. Trained on 215 million (question, answer) pairs, it is optimized for tasks involving sentence similarity and feature extraction.

Architecture

The model is based on the mpnet-base architecture. It uses mean pooling to generate sentence embeddings, which are normalized to ensure that dot-product and cosine-similarity computations are equivalent. This model is suitable for semantic search, encoding queries and paragraphs in a dense vector space to find relevant documents.

Training

The model was fine-tuned using the MultipleNegativesRankingLoss method, leveraging a combination of datasets that include WikiAnswers, PAQ, Stack Exchange, and more. A total of approximately 215 million (question, answer) pairs were used, with a focus on diverse data sources to enhance model performance in semantic search.

Guide: Running Locally

1. Install Dependencies

   • Ensure that you have sentence-transformers installed:

     pip install -U sentence-transformers
     

2. Load and Use the Model

   • Using Sentence Transformers:

     from sentence_transformers import SentenceTransformer, util
     model = SentenceTransformer('sentence-transformers/multi-qa-mpnet-base-cos-v1')
     query_emb = model.encode("Your query here")
     doc_emb = model.encode(["Your document here"])
     scores = util.dot_score(query_emb, doc_emb)[0].cpu().tolist()
     

   • Without Sentence Transformers:

     from transformers import AutoTokenizer, AutoModel
     import torch
     tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/multi-qa-mpnet-base-cos-v1")
     model = AutoModel.from_pretrained("sentence-transformers/multi-qa-mpnet-base-cos-v1")
     # Define mean pooling function and encode as shown in the detailed usage section.
     

3. Hardware Suggestions

   • For optimal performance, particularly for large datasets, consider using cloud GPUs such as those offered by Google Cloud, AWS, or Azure.

License

The model and associated code are available under the Apache 2.0 license. Please review the license terms for more detailed information on usage and redistribution.